<!doctype html>
<html style='font-size:20px !important'>
<head>
<meta charset='UTF-8'><meta name='viewport' content='width=device-width initial-scale=1'>
<title>8.chapter_eight</title><style type='text/css'>html {overflow-x: initial !important;}:root { --bg-color:#ffffff; --text-color:#333333; --select-text-bg-color:#B5D6FC; --select-text-font-color:auto; --monospace:"Lucida Console",Consolas,"Courier",monospace; }
html { font-size: 14px; background-color: var(--bg-color); color: var(--text-color); font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; -webkit-font-smoothing: antialiased; }
body { margin: 0px; padding: 0px; height: auto; bottom: 0px; top: 0px; left: 0px; right: 0px; font-size: 1rem; line-height: 1.42857; overflow-x: hidden; background: inherit; tab-size: 4; }
iframe { margin: auto; }
a.url { word-break: break-all; }
a:active, a:hover { outline: 0px; }
.in-text-selection, ::selection { text-shadow: none; background: var(--select-text-bg-color); color: var(--select-text-font-color); }
#write { margin: 0px auto; height: auto; width: inherit; word-break: normal; overflow-wrap: break-word; position: relative; white-space: normal; overflow-x: visible; padding-top: 40px; }
#write.first-line-indent p { text-indent: 2em; }
#write.first-line-indent li p, #write.first-line-indent p * { text-indent: 0px; }
#write.first-line-indent li { margin-left: 2em; }
.for-image #write { padding-left: 8px; padding-right: 8px; }
body.typora-export { padding-left: 30px; padding-right: 30px; }
.typora-export .footnote-line, .typora-export li, .typora-export p { white-space: pre-wrap; }
.typora-export .task-list-item input { pointer-events: none; }
@media screen and (max-width: 500px) {
  body.typora-export { padding-left: 0px; padding-right: 0px; }
  #write { padding-left: 20px; padding-right: 20px; }
  .CodeMirror-sizer { margin-left: 0px !important; }
  .CodeMirror-gutters { display: none !important; }
}
#write li > figure:last-child { margin-bottom: 0.5rem; }
#write ol, #write ul { position: relative; }
img { max-width: 100%; vertical-align: middle; image-orientation: from-image; }
button, input, select, textarea { color: inherit; font: inherit; }
input[type="checkbox"], input[type="radio"] { line-height: normal; padding: 0px; }
*, ::after, ::before { box-sizing: border-box; }
#write h1, #write h2, #write h3, #write h4, #write h5, #write h6, #write p, #write pre { width: inherit; }
#write h1, #write h2, #write h3, #write h4, #write h5, #write h6, #write p { position: relative; }
p { line-height: inherit; }
h1, h2, h3, h4, h5, h6 { break-after: avoid-page; break-inside: avoid; orphans: 4; }
p { orphans: 4; }
h1 { font-size: 2rem; }
h2 { font-size: 1.8rem; }
h3 { font-size: 1.6rem; }
h4 { font-size: 1.4rem; }
h5 { font-size: 1.2rem; }
h6 { font-size: 1rem; }
.md-math-block, .md-rawblock, h1, h2, h3, h4, h5, h6, p { margin-top: 1rem; margin-bottom: 1rem; }
.hidden { display: none; }
.md-blockmeta { color: rgb(204, 204, 204); font-weight: 700; font-style: italic; }
a { cursor: pointer; }
sup.md-footnote { padding: 2px 4px; background-color: rgba(238, 238, 238, 0.7); color: rgb(85, 85, 85); border-radius: 4px; cursor: pointer; }
sup.md-footnote a, sup.md-footnote a:hover { color: inherit; text-transform: inherit; text-decoration: inherit; }
#write input[type="checkbox"] { cursor: pointer; width: inherit; height: inherit; }
figure { overflow-x: auto; margin: 1.2em 0px; max-width: calc(100% + 16px); padding: 0px; }
figure > table { margin: 0px; }
tr { break-inside: avoid; break-after: auto; }
thead { display: table-header-group; }
table { border-collapse: collapse; border-spacing: 0px; width: 100%; overflow: auto; break-inside: auto; text-align: left; }
table.md-table td { min-width: 32px; }
.CodeMirror-gutters { border-right: 0px; background-color: inherit; }
.CodeMirror-linenumber { user-select: none; }
.CodeMirror { text-align: left; }
.CodeMirror-placeholder { opacity: 0.3; }
.CodeMirror pre { padding: 0px 4px; }
.CodeMirror-lines { padding: 0px; }
div.hr:focus { cursor: none; }
#write pre { white-space: pre-wrap; }
#write.fences-no-line-wrapping pre { white-space: pre; }
#write pre.ty-contain-cm { white-space: normal; }
.CodeMirror-gutters { margin-right: 4px; }
.md-fences { font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; overflow: visible; white-space: pre; background: inherit; position: relative !important; }
.md-diagram-panel { width: 100%; margin-top: 10px; text-align: center; padding-top: 0px; padding-bottom: 8px; overflow-x: auto; }
#write .md-fences.mock-cm { white-space: pre-wrap; }
.md-fences.md-fences-with-lineno { padding-left: 0px; }
#write.fences-no-line-wrapping .md-fences.mock-cm { white-space: pre; overflow-x: auto; }
.md-fences.mock-cm.md-fences-with-lineno { padding-left: 8px; }
.CodeMirror-line, twitterwidget { break-inside: avoid; }
.footnotes { opacity: 0.8; font-size: 0.9rem; margin-top: 1em; margin-bottom: 1em; }
.footnotes + .footnotes { margin-top: 0px; }
.md-reset { margin: 0px; padding: 0px; border: 0px; outline: 0px; vertical-align: top; background: 0px 0px; text-decoration: none; text-shadow: none; float: none; position: static; width: auto; height: auto; white-space: nowrap; cursor: inherit; -webkit-tap-highlight-color: transparent; line-height: normal; font-weight: 400; text-align: left; box-sizing: content-box; direction: ltr; }
li div { padding-top: 0px; }
blockquote { margin: 1rem 0px; }
li .mathjax-block, li p { margin: 0.5rem 0px; }
li { margin: 0px; position: relative; }
blockquote > :last-child { margin-bottom: 0px; }
blockquote > :first-child, li > :first-child { margin-top: 0px; }
.footnotes-area { color: rgb(136, 136, 136); margin-top: 0.714rem; padding-bottom: 0.143rem; white-space: normal; }
#write .footnote-line { white-space: pre-wrap; }
@media print {
  body, html { border: 1px solid transparent; height: 99%; break-after: avoid; break-before: avoid; font-variant-ligatures: no-common-ligatures; }
  #write { margin-top: 0px; padding-top: 0px; border-color: transparent !important; }
  .typora-export * { -webkit-print-color-adjust: exact; }
  html.blink-to-pdf { font-size: 13px; }
  .typora-export #write { break-after: avoid; }
  .typora-export #write::after { height: 0px; }
  .is-mac table { break-inside: avoid; }
}
.footnote-line { margin-top: 0.714em; font-size: 0.7em; }
a img, img a { cursor: pointer; }
pre.md-meta-block { font-size: 0.8rem; min-height: 0.8rem; white-space: pre-wrap; background: rgb(204, 204, 204); display: block; overflow-x: hidden; }
p > .md-image:only-child:not(.md-img-error) img, p > img:only-child { display: block; margin: auto; }
#write.first-line-indent p > .md-image:only-child:not(.md-img-error) img { left: -2em; position: relative; }
p > .md-image:only-child { display: inline-block; width: 100%; }
#write .MathJax_Display { margin: 0.8em 0px 0px; }
.md-math-block { width: 100%; }
.md-math-block:not(:empty)::after { display: none; }
[contenteditable="true"]:active, [contenteditable="true"]:focus, [contenteditable="false"]:active, [contenteditable="false"]:focus { outline: 0px; box-shadow: none; }
.md-task-list-item { position: relative; list-style-type: none; }
.task-list-item.md-task-list-item { padding-left: 0px; }
.md-task-list-item > input { position: absolute; top: 0px; left: 0px; margin-left: -1.2em; margin-top: calc(1em - 10px); border: none; }
.math { font-size: 1rem; }
.md-toc { min-height: 3.58rem; position: relative; font-size: 0.9rem; border-radius: 10px; }
.md-toc-content { position: relative; margin-left: 0px; }
.md-toc-content::after, .md-toc::after { display: none; }
.md-toc-item { display: block; color: rgb(65, 131, 196); }
.md-toc-item a { text-decoration: none; }
.md-toc-inner:hover { text-decoration: underline; }
.md-toc-inner { display: inline-block; cursor: pointer; }
.md-toc-h1 .md-toc-inner { margin-left: 0px; font-weight: 700; }
.md-toc-h2 .md-toc-inner { margin-left: 2em; }
.md-toc-h3 .md-toc-inner { margin-left: 4em; }
.md-toc-h4 .md-toc-inner { margin-left: 6em; }
.md-toc-h5 .md-toc-inner { margin-left: 8em; }
.md-toc-h6 .md-toc-inner { margin-left: 10em; }
@media screen and (max-width: 48em) {
  .md-toc-h3 .md-toc-inner { margin-left: 3.5em; }
  .md-toc-h4 .md-toc-inner { margin-left: 5em; }
  .md-toc-h5 .md-toc-inner { margin-left: 6.5em; }
  .md-toc-h6 .md-toc-inner { margin-left: 8em; }
}
a.md-toc-inner { font-size: inherit; font-style: inherit; font-weight: inherit; line-height: inherit; }
.footnote-line a:not(.reversefootnote) { color: inherit; }
.md-attr { display: none; }
.md-fn-count::after { content: "."; }
code, pre, samp, tt { font-family: var(--monospace); }
kbd { margin: 0px 0.1em; padding: 0.1em 0.6em; font-size: 0.8em; color: rgb(36, 39, 41); background: rgb(255, 255, 255); border: 1px solid rgb(173, 179, 185); border-radius: 3px; box-shadow: rgba(12, 13, 14, 0.2) 0px 1px 0px, rgb(255, 255, 255) 0px 0px 0px 2px inset; white-space: nowrap; vertical-align: middle; }
.md-comment { color: rgb(162, 127, 3); opacity: 0.8; font-family: var(--monospace); }
code { text-align: left; vertical-align: initial; }
a.md-print-anchor { white-space: pre !important; border-width: initial !important; border-style: none !important; border-color: initial !important; display: inline-block !important; position: absolute !important; width: 1px !important; right: 0px !important; outline: 0px !important; background: 0px 0px !important; text-decoration: initial !important; text-shadow: initial !important; }
.md-inline-math .MathJax_SVG .noError { display: none !important; }
.html-for-mac .inline-math-svg .MathJax_SVG { vertical-align: 0.2px; }
.md-math-block .MathJax_SVG_Display { text-align: center; margin: 0px; position: relative; text-indent: 0px; max-width: none; max-height: none; min-height: 0px; min-width: 100%; width: auto; overflow-y: hidden; display: block !important; }
.MathJax_SVG_Display, .md-inline-math .MathJax_SVG_Display { width: auto; margin: inherit; display: inline-block !important; }
.MathJax_SVG .MJX-monospace { font-family: var(--monospace); }
.MathJax_SVG .MJX-sans-serif { font-family: sans-serif; }
.MathJax_SVG { display: inline; font-style: normal; font-weight: 400; line-height: normal; zoom: 90%; text-indent: 0px; text-align: left; text-transform: none; letter-spacing: normal; word-spacing: normal; overflow-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; }
.MathJax_SVG * { transition: none 0s ease 0s; }
.MathJax_SVG_Display svg { vertical-align: middle !important; margin-bottom: 0px !important; margin-top: 0px !important; }
.os-windows.monocolor-emoji .md-emoji { font-family: "Segoe UI Symbol", sans-serif; }
.md-diagram-panel > svg { max-width: 100%; }
[lang="flow"] svg, [lang="mermaid"] svg { max-width: 100%; height: auto; }
[lang="mermaid"] .node text { font-size: 1rem; }
table tr th { border-bottom: 0px; }
video { max-width: 100%; display: block; margin: 0px auto; }
iframe { max-width: 100%; width: 100%; border: none; }
.highlight td, .highlight tr { border: 0px; }
svg[id^="mermaidChart"] { line-height: 1em; }
mark { background: rgb(255, 255, 0); color: rgb(0, 0, 0); }
.md-html-inline .md-plain, .md-html-inline strong, mark .md-inline-math, mark strong { color: inherit; }
mark .md-meta { color: rgb(0, 0, 0); opacity: 0.3 !important; }
@media print {
  .typora-export h1, .typora-export h2, .typora-export h3, .typora-export h4, .typora-export h5, .typora-export h6 { break-inside: avoid; }
}


/* Flowchart variables */
/* Sequence Diagram variables */
/* Gantt chart variables */
/* state colors */
.label {
  
  color: #333; }

.label text {
  fill: #333; }

.node rect,
.node circle,
.node ellipse,
.node polygon {
  fill: #BDD5EA;
  stroke: #9370DB;
  stroke-width: 1px; }

.node .label {
  text-align: center; }

.node.clickable {
  cursor: pointer; }

.arrowheadPath {
  fill: lightgrey; }

.edgePath .path {
  stroke: lightgrey;
  stroke-width: 1.5px; }

.edgeLabel {
  background-color: #e8e8e8;
  text-align: center; }

.cluster rect {
  fill: #6D6D65;
  stroke: rgba(255, 255, 255, 0.25);
  stroke-width: 1px; }

.cluster text {
  fill: #F9FFFE; }

div.mermaidTooltip {
  position: absolute;
  text-align: center;
  max-width: 200px;
  padding: 2px;
  
  font-size: 12px;
  background: #6D6D65;
  border: 1px solid rgba(255, 255, 255, 0.25);
  border-radius: 2px;
  pointer-events: none;
  z-index: 100; }

.actor {
  stroke: #81B1DB;
  fill: #BDD5EA; }

text.actor {
  fill: black;
  stroke: none; }

.actor-line {
  stroke: lightgrey; }

.messageLine0 {
  stroke-width: 1.5;
  stroke-dasharray: '2 2';
  stroke: lightgrey; }

.messageLine1 {
  stroke-width: 1.5;
  stroke-dasharray: '2 2';
  stroke: lightgrey; }

#arrowhead {
  fill: lightgrey; }

.sequenceNumber {
  fill: white; }

#sequencenumber {
  fill: lightgrey; }

#crosshead path {
  fill: lightgrey !important;
  stroke: lightgrey !important; }

.messageText {
  fill: lightgrey;
  stroke: none; }

.labelBox {
  stroke: #81B1DB;
  fill: #BDD5EA; }

.labelText {
  fill: #323D47;
  stroke: none; }

.loopText {
  fill: lightgrey;
  stroke: none; }

.loopLine {
  stroke-width: 2;
  stroke-dasharray: '2 2';
  stroke: #81B1DB; }

.note {
  stroke: rgba(255, 255, 255, 0.25);
  fill: #fff5ad; }

.noteText {
  fill: black;
  stroke: none;
  
  font-size: 14px; }

.activation0 {
  fill: #f4f4f4;
  stroke: #666; }

.activation1 {
  fill: #f4f4f4;
  stroke: #666; }

.activation2 {
  fill: #f4f4f4;
  stroke: #666; }

/** Section styling */
.section {
  stroke: none;
  opacity: 0.2; }

.section0 {
  fill: rgba(255, 255, 255, 0.3); }

.section2 {
  fill: #EAE8B9; }

.section1,
.section3 {
  fill: white;
  opacity: 0.2; }

.sectionTitle0 {
  fill: #F9FFFE; }

.sectionTitle1 {
  fill: #F9FFFE; }

.sectionTitle2 {
  fill: #F9FFFE; }

.sectionTitle3 {
  fill: #F9FFFE; }

.sectionTitle {
  text-anchor: start;
  font-size: 11px;
  text-height: 14px;
   }

/* Grid and axis */
.grid .tick {
  stroke: lightgrey;
  opacity: 0.3;
  shape-rendering: crispEdges; }

.grid path {
  stroke-width: 0; }

/* Today line */
.today {
  fill: none;
  stroke: #DB5757;
  stroke-width: 2px; }

/* Task styling */
/* Default task */
.task {
  stroke-width: 2; }

.taskText {
  text-anchor: middle;
   }

.taskText:not([font-size]) {
  font-size: 11px; }

.taskTextOutsideRight {
  fill: #323D47;
  text-anchor: start;
  font-size: 11px;
   }

.taskTextOutsideLeft {
  fill: #323D47;
  text-anchor: end;
  font-size: 11px; }

/* Special case clickable */
.task.clickable {
  cursor: pointer; }

.taskText.clickable {
  cursor: pointer;
  fill: #003163 !important;
  font-weight: bold; }

.taskTextOutsideLeft.clickable {
  cursor: pointer;
  fill: #003163 !important;
  font-weight: bold; }

.taskTextOutsideRight.clickable {
  cursor: pointer;
  fill: #003163 !important;
  font-weight: bold; }

/* Specific task settings for the sections*/
.taskText0,
.taskText1,
.taskText2,
.taskText3 {
  fill: #323D47; }

.task0,
.task1,
.task2,
.task3 {
  fill: #BDD5EA;
  stroke: rgba(255, 255, 255, 0.5); }

.taskTextOutside0,
.taskTextOutside2 {
  fill: lightgrey; }

.taskTextOutside1,
.taskTextOutside3 {
  fill: lightgrey; }

/* Active task */
.active0,
.active1,
.active2,
.active3 {
  fill: #81B1DB;
  stroke: rgba(255, 255, 255, 0.5); }

.activeText0,
.activeText1,
.activeText2,
.activeText3 {
  fill: #323D47 !important; }

/* Completed task */
.done0,
.done1,
.done2,
.done3 {
  stroke: grey;
  fill: lightgrey;
  stroke-width: 2; }

.doneText0,
.doneText1,
.doneText2,
.doneText3 {
  fill: #323D47 !important; }

/* Tasks on the critical line */
.crit0,
.crit1,
.crit2,
.crit3 {
  stroke: #E83737;
  fill: #E83737;
  stroke-width: 2; }

.activeCrit0,
.activeCrit1,
.activeCrit2,
.activeCrit3 {
  stroke: #E83737;
  fill: #81B1DB;
  stroke-width: 2; }

.doneCrit0,
.doneCrit1,
.doneCrit2,
.doneCrit3 {
  stroke: #E83737;
  fill: lightgrey;
  stroke-width: 2;
  cursor: pointer;
  shape-rendering: crispEdges; }

.milestone {
  transform: rotate(45deg) scale(0.8, 0.8); }

.milestoneText {
  font-style: italic; }

.doneCritText0,
.doneCritText1,
.doneCritText2,
.doneCritText3 {
  fill: #323D47 !important; }

.activeCritText0,
.activeCritText1,
.activeCritText2,
.activeCritText3 {
  fill: #323D47 !important; }

.titleText {
  text-anchor: middle;
  font-size: 18px;
  fill: #323D47;
   }

g.classGroup text {
  fill: #9370DB;
  stroke: none;
  
  font-size: 10px; }
  g.classGroup text .title {
    font-weight: bolder; }

g.classGroup rect {
  fill: #BDD5EA;
  stroke: #9370DB; }

g.classGroup line {
  stroke: #9370DB;
  stroke-width: 1; }

.classLabel .box {
  stroke: none;
  stroke-width: 0;
  fill: #BDD5EA;
  opacity: 0.5; }

.classLabel .label {
  fill: #9370DB;
  font-size: 10px; }

.relation {
  stroke: #9370DB;
  stroke-width: 1;
  fill: none; }

#compositionStart {
  fill: #9370DB;
  stroke: #9370DB;
  stroke-width: 1; }

#compositionEnd {
  fill: #9370DB;
  stroke: #9370DB;
  stroke-width: 1; }

#aggregationStart {
  fill: #BDD5EA;
  stroke: #9370DB;
  stroke-width: 1; }

#aggregationEnd {
  fill: #BDD5EA;
  stroke: #9370DB;
  stroke-width: 1; }

#dependencyStart {
  fill: #9370DB;
  stroke: #9370DB;
  stroke-width: 1; }

#dependencyEnd {
  fill: #9370DB;
  stroke: #9370DB;
  stroke-width: 1; }

#extensionStart {
  fill: #9370DB;
  stroke: #9370DB;
  stroke-width: 1; }

#extensionEnd {
  fill: #9370DB;
  stroke: #9370DB;
  stroke-width: 1; }

.commit-id,
.commit-msg,
.branch-label {
  fill: lightgrey;
  color: lightgrey;
   }

.pieTitleText {
  text-anchor: middle;
  font-size: 25px;
  fill: #eee;
}

g.stateGroup text {
  stroke: none;
  font-size: 10px;
}

g.stateGroup circle {
  fill: white !important;
  stroke: white !important;
}

g.stateGroup .state-title {
  font-weight: bolder;
  fill: black; }

g.stateGroup rect {
  fill: #ececff;
  stroke: #9370DB; }

g.stateGroup line {
  stroke: #9370DB;
  stroke-width: 1; }

.transition {
  stroke: #9370DB;
  stroke-width: 1;
  fill: none; }

.stateGroup .composit {
  fill: #555;
  border-bottom: 1px; }

.state-note {
  stroke: rgba(255, 255, 255, 0.25);
  fill: #fff5ad; }
  .state-note text {
    fill: black;
    stroke: none;
    font-size: 10px; }

.stateLabel .box {
  stroke: none;
  stroke-width: 0;
  fill: #BDD5EA;
  opacity: 0.5; }

.stateLabel text {
  fill: black;
  font-size: 10px;
  font-weight: bold;
}

.cluster-label {
  color:black;
}

.statediagram-cluster rect {
  fill: #BDD5EA;
  stroke: #9370DB; 
  stroke-width: 1px;
}
.statediagram-cluster rect.outer {
  rx: 5px;
  ry: 5px;
}
.statediagram-state .divider {
  stroke: #9370DB; 
}

.statediagram-state .title-state {
  rx: 5px;
  ry: 5px;
}
.statediagram-cluster.statediagram-cluster .inner {
  fill: white;
}
.statediagram-cluster.statediagram-cluster-alt .inner {
  fill: #e0e0e0;
}

.statediagram-cluster .inner {
  rx:0;
  ry:0;
}

.statediagram-state rect.basic {
  rx: 5px;
  ry: 5px;
}
.statediagram-state rect.divider {
  stroke-dasharray: 10,10;
  fill: #efefef;
}

.note-edge {
  stroke-dasharray: 5;
}

.statediagram-note rect {
  stroke: var(--cluster-border);
  fill: #fff5ad;
  stroke-width: 1px;
  rx: 0;
  ry: 0;
}

.node circle.state-start {
  fill: black;
  stroke: black;
}
.node circle.state-end {
  fill: black;
  stroke: white;
  stroke-width: 1.5
}
#statediagram-barbEnd {
  fill: #9370DB; 
}

/* CSS Document */

/** code highlight */

.cm-s-inner .cm-variable,
.cm-s-inner .cm-operator,
.cm-s-inner .cm-property {
    color: #b8bfc6;
}

.cm-s-inner .cm-keyword {
    color: #C88FD0;
}

.cm-s-inner .cm-tag {
    color: #7DF46A;
}

.cm-s-inner .cm-attribute {
    color: #7575E4;
}

.CodeMirror div.CodeMirror-cursor {
    border-left: 1px solid #b8bfc6;
    z-index: 3;
}

.cm-s-inner .cm-string {
    color: #D26B6B;
}

.cm-s-inner .cm-comment,
.cm-s-inner.cm-comment {
    color: #DA924A;
}

.cm-s-inner .cm-header,
.cm-s-inner .cm-def,
.cm-s-inner.cm-header,
.cm-s-inner.cm-def {
    color: #8d8df0;
}

.cm-s-inner .cm-quote,
.cm-s-inner.cm-quote {
    color: #57ac57;
}

.cm-s-inner .cm-hr {
    color: #d8d5d5;
}

.cm-s-inner .cm-link {
    color: #d3d3ef;
}

.cm-s-inner .cm-negative {
    color: #d95050;
}

.cm-s-inner .cm-positive {
    color: #50e650;
}

.cm-s-inner .cm-string-2 {
    color: #f50;
}

.cm-s-inner .cm-meta,
.cm-s-inner .cm-qualifier {
    color: #b7b3b3;
}

.cm-s-inner .cm-builtin {
    color: #f3b3f8;
}

.cm-s-inner .cm-bracket {
    color: #997;
}

.cm-s-inner .cm-atom,
.cm-s-inner.cm-atom {
    color: #84B6CB;
}

.cm-s-inner .cm-number {
    color: #64AB8F;
}

.cm-s-inner .cm-variable {
    color: #b8bfc6;
}

.cm-s-inner .cm-variable-2 {
    color: #9FBAD5;
}

.cm-s-inner .cm-variable-3 {
    color: #1cc685;
}

.CodeMirror-selectedtext,
.CodeMirror-selected {
    background: #4a89dc;
    color: #fff !important;
    text-shadow: none;
}

.CodeMirror-gutters {
    border-right: none;
}

/* CSS Document */

/** markdown source **/
.cm-s-typora-default .cm-header, 
.cm-s-typora-default .cm-property
{
    color: #cebcca;
}

.CodeMirror.cm-s-typora-default div.CodeMirror-cursor{
    border-left: 3px solid #b8bfc6;
}

.cm-s-typora-default .cm-comment {
    color: #9FB1FF;
}

.cm-s-typora-default .cm-string {
    color: #A7A7D9
}

.cm-s-typora-default .cm-atom, .cm-s-typora-default .cm-number {
    color: #848695;
    font-style: italic;
}

.cm-s-typora-default .cm-link {
    color: #95B94B;
}

.cm-s-typora-default .CodeMirror-activeline-background {
    background: rgba(51, 51, 51, 0.72);
}

.cm-s-typora-default .cm-comment, .cm-s-typora-default .cm-code {
	color: #8aa1e1;
}@import "";
@import "";
@import "";

:root {
    --bg-color:  #363B40;
    --side-bar-bg-color: #2E3033;
    --text-color: #b8bfc6;

    --select-text-bg-color:#4a89dc;

    --item-hover-bg-color: #0a0d16;
    --control-text-color: #b7b7b7;
    --control-text-hover-color: #eee;
    --window-border: 1px solid #555;

    --active-file-bg-color: rgb(34, 34, 34);
    --active-file-border-color: #8d8df0;

    --primary-color: #a3d5fe;

    --active-file-text-color: white;
    --item-hover-bg-color: #70717d;
    --item-hover-text-color: white;
    --primary-color: #6dc1e7;

    --rawblock-edit-panel-bd: #333;

    --search-select-bg-color: #428bca;
}

html {
    font-size: 16px;
}

html,
body {
    -webkit-text-size-adjust: 100%;
    -ms-text-size-adjust: 100%;
    background: #363B40;
    background: var(--bg-color);
    fill: currentColor;
    line-height: 1.625rem;
}

#write {
    max-width: 1080px;
}


@media only screen and (min-width: 1400px) {
	#write {
		max-width: 1024px;
	}
}

@media only screen and (min-width: 1800px) {
	#write {
		max-width: 1200px;
	}
}

html,
body,
button,
input,
select,
textarea,
div.code-tooltip-content {
    color: #b8bfc6;
    border-color: transparent;
}

div.code-tooltip,
.md-hover-tip .md-arrow:after {
    background: #333;
}

.popover.bottom > .arrow:after {
    border-bottom-color: #333;
}

html,
body,
button,
input,
select,
textarea {
    font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
}

hr {
    height: 2px;
    border: 0;
    margin: 24px 0 !important;
}

h1,
h2,
h3,
h4,
h5,
h6 {
    font-family: "Lucida Grande", "Corbel", sans-serif;
    font-weight: normal;
    clear: both;
    -ms-word-wrap: break-word;
    word-wrap: break-word;
    margin: 0;
    padding: 0;
    color: #DEDEDE
}

h1 {
    font-size: 2.5rem;
    /* 36px */
    line-height: 2.75rem;
    /* 40px */
    margin-bottom: 1.5rem;
    /* 24px */
    letter-spacing: -1.5px;
}

h2 {
    font-size: 1.63rem;
    /* 24px */
    line-height: 1.875rem;
    /* 30px */
    margin-bottom: 1.5rem;
    /* 24px */
    letter-spacing: -1px;
    font-weight: bold;
}

h3 {
    font-size: 1.17rem;
    /* 18px */
    line-height: 1.5rem;
    /* 24px */
    margin-bottom: 1.5rem;
    /* 24px */
    letter-spacing: -1px;
    font-weight: bold;
}

h4 {
    font-size: 1.12rem;
    /* 16px */
    line-height: 1.375rem;
    /* 22px */
    margin-bottom: 1.5rem;
    /* 24px */
    color: white;
}

h5 {
    font-size: 0.97rem;
    /* 16px */
    line-height: 1.25rem;
    /* 22px */
    margin-bottom: 1.5rem;
    /* 24px */
    font-weight: bold;
}

h6 {
    font-size: 0.93rem;
    /* 16px */
    line-height: 1rem;
    /* 16px */
    margin-bottom: 0.75rem;
    color: white;
}

@media (min-width: 980px) {
    h3.md-focus:before,
    h4.md-focus:before,
    h5.md-focus:before,
    h6.md-focus:before {
        color: #ddd;
        border: 1px solid #ddd;
        border-radius: 3px;
        position: absolute;
        left: -1.642857143rem;
        top: .357142857rem;
        float: left;
        font-size: 9px;
        padding-left: 2px;
        padding-right: 2px;
        vertical-align: bottom;
        font-weight: normal;
        line-height: normal;
    }

    h3.md-focus:before {
        content: 'h3';
    }

    h4.md-focus:before {
        content: 'h4';
    }

    h5.md-focus:before {
        content: 'h5';
        top: 0px;
    }

    h6.md-focus:before {
        content: 'h6';
        top: 0px;
    }
}

a {
    text-decoration: none;
    outline: 0;
}

a:hover {
    outline: 0;
}

a:focus {
    outline: thin dotted;
}

sup.md-footnote {
    background-color: #555;
    color: #ddd;
}

p {
    -ms-word-wrap: break-word;
    word-wrap: break-word;
}

p,
ul,
dd,
ol,
hr,
address,
pre,
table,
iframe,
.wp-caption,
.wp-audio-shortcode,
.wp-video-shortcode {
    margin-top: 0;
    margin-bottom: 1.5rem;
    /* 24px */
}

li > blockquote {
	margin-bottom: 0;
}

audio:not([controls]) {
    display: none;
}

[hidden] {
    display: none;
}

::-moz-selection {
    background: #4a89dc;
    color: #fff;
    text-shadow: none;
}

*.in-text-selection,
::selection {
    background: #4a89dc;
    color: #fff;
    text-shadow: none;
}

ul,
ol {
    padding: 0 0 0 1.875rem;
    /* 30px */
}

ul {
    list-style: square;
}

ol {
    list-style: decimal;
}

ul ul,
ol ol,
ul ol,
ol ul {
    margin: 0;
}

b,
th,
dt,
strong {
    font-weight: bold;
}

i,
em,
dfn,
cite {
    font-style: italic;
}

blockquote {
    padding-left: 1.875rem;
    margin: 0 0 1.875rem 1.875rem;
    border-left: solid 2px #474d54;
    padding-left: 30px;
    margin-top: 35px;
}

pre,
code,
kbd,
tt,
var {
    font-size: 0.875rem;
    font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace;
}

code,
tt,
var {
    background: rgba(0, 0, 0, 0.05);
}

kbd {
    padding: 2px 4px;
    font-size: 90%;
    color: #fff;
    background-color: #333;
    border-radius: 3px;
    box-shadow: inset 0 -1px 0 rgba(0,0,0,.25);
}

pre.md-fences {
    padding: 10px 10px 10px 30px;
    margin-bottom: 20px;
    background: #333;
}

.CodeMirror-gutters {
    background: #333;
    border-right: 1px solid transparent;
}

.enable-diagrams pre.md-fences[lang="sequence"] .code-tooltip,
.enable-diagrams pre.md-fences[lang="flow"] .code-tooltip,
.enable-diagrams pre.md-fences[lang="mermaid"] .code-tooltip {
    bottom: -2.2em;
    right: 4px;
}

code,
kbd,
tt,
var {
    padding: 2px 5px;
}

table {
    max-width: 100%;
    width: 100%;
    border-collapse: collapse;
    border-spacing: 0;
}

th,
td {
    padding: 5px 10px;
    vertical-align: top;
}

a {
    -webkit-transition: all .2s ease-in-out;
    transition: all .2s ease-in-out;
}

hr {
    background: #474d54;
    /* variable */
}

h1 {
    margin-top: 2em;
}

a {
    color: #e0e0e0;
    text-decoration: underline;
}

a:hover {
    color: #fff;
}

.md-inline-math script {
    color: #81b1db;
}

b,
th,
dt,
strong {
    color: #DEDEDE;
    /* variable */
}

mark {
    background: #D3D40E;
}

blockquote {
    color: #9DA2A6;
}

table a {
    color: #DEDEDE;
    /* variable */
}

th,
td {
    border: solid 1px #474d54;
    /* variable */
}

.task-list {
    padding-left: 0;
}

.md-task-list-item {
    padding-left: 1.25rem;
}

.md-task-list-item > input {
    top: auto;
}

.md-task-list-item > input:before {
    content: "";
    display: inline-block;
    width: 0.875rem;
    height: 0.875rem;
    vertical-align: middle;
    text-align: center;
    border: 1px solid #b8bfc6;
    background-color: #363B40;
    margin-top: -0.4rem;
}

.md-task-list-item > input:checked:before,
.md-task-list-item > input[checked]:before {
    content: '\221A';
    /*◘*/
    font-size: 0.625rem;
    line-height: 0.625rem;
    color: #DEDEDE;
}

/** quick open **/
.auto-suggest-container {
    border: 0px;
    background-color: #525C65;
}

#typora-quick-open {
    background-color: #525C65;
}

#typora-quick-open input{
    background-color: #525C65;
    border: 0;
    border-bottom: 1px solid grey;
}

.typora-quick-open-item {
    background-color: inherit;
    color: inherit;
}

.typora-quick-open-item.active,
.typora-quick-open-item:hover {
    background-color: #4D8BDB;
    color: white;
}

.typora-quick-open-item:hover {
    background-color: rgba(77, 139, 219, 0.8);
}

.typora-search-spinner > div {
  background-color: #fff;
}

#write pre.md-meta-block {
    border-bottom: 1px dashed #ccc;
    background: transparent;
    padding-bottom: 0.6em;
    line-height: 1.6em;
}

.btn,
.btn .btn-default {
    background: transparent;
    color: #b8bfc6;
}

.ty-table-edit {
    border-top: 1px solid gray;
    background-color: #363B40;
}

.popover-title {
    background: transparent;
}

.md-image>.md-meta {
    color: #BBBBBB;
    background: transparent;
}

.md-expand.md-image>.md-meta {
    color: #DDD;
}

#write>h3:before,
#write>h4:before,
#write>h5:before,
#write>h6:before {
    border: none;
    border-radius: 0px;
    color: #888;
    text-decoration: underline;
    left: -1.4rem;
    top: 0.2rem;
}

#write>h3.md-focus:before {
    top: 2px;
}

#write>h4.md-focus:before {
    top: 2px;
}

.md-toc-item {
    color: #A8C2DC;
}

#write div.md-toc-tooltip {
    background-color: #363B40;
}

.dropdown-menu .btn:hover,
.dropdown-menu .btn:focus,
.md-toc .btn:hover,
.md-toc .btn:focus {
    color: white;
    background: black;
}

#toc-dropmenu {
    background: rgba(50, 54, 59, 0.93);
    border: 1px solid rgba(253, 253, 253, 0.15);
}

#toc-dropmenu .divider {
    background-color: #9b9b9b;
}

.outline-expander:before {
    top: 2px;
}

#typora-sidebar {
    box-shadow: none;
    border-right: 1px dashed;
    border-right: none;
}

.sidebar-tabs {
    border-bottom:0;
}

#typora-sidebar:hover .outline-title-wrapper {
    border-left: 1px dashed;
}

.outline-title-wrapper .btn {
    color: inherit;
}

.outline-item:hover {
    border-color: #363B40;
    background-color: #363B40;
    color: white;
}

h1.md-focus .md-attr,
h2.md-focus .md-attr,
h3.md-focus .md-attr,
h4.md-focus .md-attr,
h5.md-focus .md-attr,
h6.md-focus .md-attr,
.md-header-span .md-attr {
    color: #8C8E92;
    display: inline;
}

.md-comment {
    color: #5a95e3;
    opacity: 1;
}

.md-inline-math svg {
    color: #b8bfc6;
}

#math-inline-preview .md-arrow:after {
    background: black;
}

.modal-content {
    background: var(--bg-color);
    border: 0;
}

.modal-title {
    font-size: 1.5em;
}

.modal-content input {
    background-color: rgba(26, 21, 21, 0.51);
    color: white;
}

.modal-content .input-group-addon {
    color: white;
}

.modal-backdrop {
    background-color: rgba(174, 174, 174, 0.7);
}

.modal-content .btn-primary {
    border-color: var(--primary-color);
}

.md-table-resize-popover {
    background-color: #333;
}

.form-inline .input-group .input-group-addon {
    color: white;
}

#md-searchpanel {
    border-bottom: 1px dashed grey;
}

/** UI for electron */

.context-menu,
#spell-check-panel,
#footer-word-count-info {
    background-color: #42464A;
}

.context-menu.dropdown-menu .divider,
.dropdown-menu .divider {
    background-color: #777777;
}

footer {
    color: inherit;
}

@media (max-width: 1000px) {
    footer {
        border-top: none;
    }
    footer:hover {
        color: inherit;
    }
}

#file-info-file-path .file-info-field-value:hover {
    background-color: #555;
    color: #dedede;
}

.megamenu-content,
.megamenu-opened header {
    background: var(--bg-color);
}

.megamenu-menu-panel h2,
.megamenu-menu-panel h1,
.long-btn {
    color: inherit;
}

.megamenu-menu-panel input[type='text'] {
    background: inherit;
    border: 0;
    border-bottom: 1px solid;
}

#recent-file-panel-action-btn {
    background: inherit;
    border: 1px grey solid;
}

.megamenu-menu-panel .dropdown-menu > li > a {
    color: inherit;
    background-color: #2F353A;
    text-decoration: none;
}

.megamenu-menu-panel table td:nth-child(1) {
    color: inherit;
    font-weight: bold;
}

.megamenu-menu-panel tbody tr:hover td:nth-child(1) {
    color: white;
}

.modal-footer .btn-default, 
.modal-footer .btn-primary,
.modal-footer .btn-default:not(:hover) {
    border: 1px solid;
    border-color: transparent;
}

.btn-default:hover, .btn-default:focus, .btn-default.focus, .btn-default:active, .btn-default.active, .open > .dropdown-toggle.btn-default {
    color: white;
    border: 1px solid #ddd;
    background-color: inherit;
}

.modal-header {
    border-bottom: 0;
}

.modal-footer {
    border-top: 0;
}

#recent-file-panel tbody tr:nth-child(2n-1) {
    background-color: transparent !important;
}

.megamenu-menu-panel tbody tr:hover td:nth-child(2) {
    color: inherit;
}

.megamenu-menu-panel .btn {
    border: 1px solid #eee;
    background: transparent;
}

.mouse-hover .toolbar-icon.btn:hover,
#w-full.mouse-hover,
#w-pin.mouse-hover {
    background-color: inherit;
}

.typora-node::-webkit-scrollbar {
    width: 5px;
}

.typora-node::-webkit-scrollbar-thumb:vertical {
    background: rgba(250, 250, 250, 0.3);
}

.typora-node::-webkit-scrollbar-thumb:vertical:active {
    background: rgba(250, 250, 250, 0.5);
}

#w-unpin {
    background-color: #4182c4;
}

#top-titlebar, #top-titlebar * {
    color: var(--item-hover-text-color);
}

.typora-sourceview-on #toggle-sourceview-btn,
#footer-word-count:hover,
.ty-show-word-count #footer-word-count {
    background: #333333;
}

#toggle-sourceview-btn:hover {
    color: #eee;
    background: #333333;
}

/** focus mode */
.on-focus-mode .md-end-block:not(.md-focus):not(.md-focus-container) * {
    color: #686868 !important;
}

.on-focus-mode .md-end-block:not(.md-focus) img,
.on-focus-mode .md-task-list-item:not(.md-focus-container)>input {
    opacity: #686868 !important;
}

.on-focus-mode li[cid]:not(.md-focus-container){
    color: #686868;
}

.on-focus-mode .md-fences.md-focus .CodeMirror-code>*:not(.CodeMirror-activeline) *,
.on-focus-mode .CodeMirror.cm-s-inner:not(.CodeMirror-focused) * {
    color: #686868 !important;
}

.on-focus-mode .md-focus,
.on-focus-mode .md-focus-container {
    color: #fff;
}

.on-focus-mode #typora-source .CodeMirror-code>*:not(.CodeMirror-activeline) * {
    color: #686868 !important;
}


/*diagrams*/
#write .md-focus .md-diagram-panel {
    border: 1px solid #ddd;
    margin-left: -1px;
    width: calc(100% + 2px);
}

/*diagrams*/
#write .md-focus.md-fences-with-lineno .md-diagram-panel {
    margin-left: auto;
}

.md-diagram-panel-error {
    color: #f1908e;
}

.active-tab-files #info-panel-tab-file,
.active-tab-files #info-panel-tab-file:hover,
.active-tab-outline #info-panel-tab-outline,
.active-tab-outline #info-panel-tab-outline:hover {
    color: #eee;
}

.sidebar-footer-item:hover,
.footer-item:hover {
    background: inherit;
    color: white;
}

.ty-side-sort-btn.active,
.ty-side-sort-btn:hover,
.selected-folder-menu-item a:after {
    color: white;
}

#sidebar-files-menu {
    border:solid 1px;
    box-shadow: 4px 4px 20px rgba(0, 0, 0, 0.79);
    background-color: var(--bg-color);
}

.file-list-item {
    border-bottom:none;
}

.file-list-item-summary {
    opacity: 1;
}

.file-list-item.active:first-child {
    border-top: none;
}

.file-node-background {
    height: 32px;
}

.file-library-node.active>.file-node-content,
.file-list-item.active {
    color: white;
    color: var(--active-file-text-color);
}

.file-library-node.active>.file-node-background{
    background-color: rgb(34, 34, 34);
    background-color: var(--active-file-bg-color);
}
.file-list-item.active {
    background-color: rgb(34, 34, 34);
    background-color: var(--active-file-bg-color);
}

#ty-tooltip {
    background-color: black;
    color: #eee;
}

.md-task-list-item>input {
    margin-left: -1.3em;
    margin-top: 0.3rem;
    -webkit-appearance: none;
}

.md-mathjax-midline {
    background-color: #57616b;
    border-bottom: none;
}

footer.ty-footer {
    border-color: #656565;
}

.ty-preferences .btn-default {
    background: transparent;
}
.ty-preferences .btn-default:hover {
    background: #57616b;
}

.ty-preferences select {
    border: 1px solid #989698;
    height: 21px;
}

.ty-preferences .nav-group-item.active {
    background: var(--item-hover-bg-color);
}

.ty-preferences input[type="search"] {
    border-color: #333;
    background: #333;
    line-height: 22px;
    border-radius: 6px;
    color: white;
}

.ty-preferences input[type="search"]:focus {
    box-shadow: none;
}

[data-is-directory="true"] .file-node-content {
    margin-bottom: 0;
}

.file-node-title {
    line-height: 22px;
}

.html-for-mac .file-node-open-state, .html-for-mac .file-node-icon {
    line-height: 26px;
}

::-webkit-scrollbar-thumb {
    background: rgba(230, 230, 230, 0.30);
}

::-webkit-scrollbar-thumb:active {
    background: rgba(230, 230, 230, 0.50);
}

#typora-sidebar:hover div.sidebar-content-content::-webkit-scrollbar-thumb:horizontal {
    background: rgba(230, 230, 230, 0.30);
}

.nav-group-item:active {
    background-color: #474d54;
}

.md-search-hit {
    background: rgba(199, 140, 60, 0.81);
    color: #eee;
}

.md-search-hit * {
    color: #eee;
}

#md-searchpanel input {
    color: white;
}

.export-detail,
.export-item.active,
.export-items-list-control {
    background: #d6d6d4
}


</style>
</head>
<body class='typora-export os-windows'>
<div id='write'  class=''><h2><a name="第八章执行者--评论者方法" class="md-header-anchor"></a><span>第八章：执行者 / 评论者方法</span></h2><p><span>将策略梯度算法和自益相结合：一方面，用含参函数近似价值函数，然后利用这个价值函数的近似值来估计回报值；另一方面，利用估计得到的回报值估计策略梯度，进而更新策略参数。这两方面又被称为</span><strong><span>评论者</span></strong><span>（critic）和</span><strong><span>执行者</span></strong><span>（actor），所以该类算法被称为</span><strong><span>执行者 / 评论者算法</span></strong><span>（actor-critic algorithm）。</span></p><h3><a name="一同策执行者--评论者算法" class="md-header-anchor"></a><span>一、同策执行者 / 评论者算法</span></h3><p><span>执行者 / 评论者算法同样使用含参函数  </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="8.618ex" height="2.71ex" viewBox="0 -832.7 3710.3 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2686-MJMATHI-68" d="M137 683Q138 683 209 688T282 694Q294 694 294 685Q294 674 258 534Q220 386 220 383Q220 381 227 388Q288 442 357 442Q411 442 444 415T478 336Q478 285 440 178T402 50Q403 36 407 31T422 26Q450 26 474 56T513 138Q516 149 519 151T535 153Q555 153 555 145Q555 144 551 130Q535 71 500 33Q466 -10 419 -10H414Q367 -10 346 17T325 74Q325 90 361 192T398 345Q398 404 354 404H349Q266 404 205 306L198 293L164 158Q132 28 127 16Q114 -11 83 -11Q69 -11 59 -2T48 16Q48 30 121 320L195 616Q195 629 188 632T149 637H128Q122 643 122 645T124 664Q129 683 137 683Z"></path><path stroke-width="0" id="E2686-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2686-MJMATHI-73" d="M131 289Q131 321 147 354T203 415T300 442Q362 442 390 415T419 355Q419 323 402 308T364 292Q351 292 340 300T328 326Q328 342 337 354T354 372T367 378Q368 378 368 379Q368 382 361 388T336 399T297 405Q249 405 227 379T204 326Q204 301 223 291T278 274T330 259Q396 230 396 163Q396 135 385 107T352 51T289 7T195 -10Q118 -10 86 19T53 87Q53 126 74 143T118 160Q133 160 146 151T160 120Q160 94 142 76T111 58Q109 57 108 57T107 55Q108 52 115 47T146 34T201 27Q237 27 263 38T301 66T318 97T323 122Q323 150 302 164T254 181T195 196T148 231Q131 256 131 289Z"></path><path stroke-width="0" id="E2686-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2686-MJMATHI-61" d="M33 157Q33 258 109 349T280 441Q331 441 370 392Q386 422 416 422Q429 422 439 414T449 394Q449 381 412 234T374 68Q374 43 381 35T402 26Q411 27 422 35Q443 55 463 131Q469 151 473 152Q475 153 483 153H487Q506 153 506 144Q506 138 501 117T481 63T449 13Q436 0 417 -8Q409 -10 393 -10Q359 -10 336 5T306 36L300 51Q299 52 296 50Q294 48 292 46Q233 -10 172 -10Q117 -10 75 30T33 157ZM351 328Q351 334 346 350T323 385T277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q217 26 254 59T298 110Q300 114 325 217T351 328Z"></path><path stroke-width="0" id="E2686-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2686-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path><path stroke-width="0" id="E2686-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2686-MJMATHI-68" x="0" y="0"></use><use xlink:href="#E2686-MJMAIN-28" x="576" y="0"></use><use xlink:href="#E2686-MJMATHI-73" x="965" y="0"></use><use xlink:href="#E2686-MJMAIN-2C" x="1434" y="0"></use><use xlink:href="#E2686-MJMATHI-61" x="1878" y="0"></use><use xlink:href="#E2686-MJMAIN-3B" x="2407" y="0"></use><use xlink:href="#E2686-MJMATHI-3B8" x="2852" y="0"></use><use xlink:href="#E2686-MJMAIN-29" x="3321" y="0"></use></g></svg></span><script type="math/tex">h(s,a;\theta)</script><span> 表示偏好，以及取 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="22.356ex" height="2.71ex" viewBox="0 -832.7 9625.4 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2687-MJMATHI-45" d="M492 213Q472 213 472 226Q472 230 477 250T482 285Q482 316 461 323T364 330H312Q311 328 277 192T243 52Q243 48 254 48T334 46Q428 46 458 48T518 61Q567 77 599 117T670 248Q680 270 683 272Q690 274 698 274Q718 274 718 261Q613 7 608 2Q605 0 322 0H133Q31 0 31 11Q31 13 34 25Q38 41 42 43T65 46Q92 46 125 49Q139 52 144 61Q146 66 215 342T285 622Q285 629 281 629Q273 632 228 634H197Q191 640 191 642T193 659Q197 676 203 680H757Q764 676 764 669Q764 664 751 557T737 447Q735 440 717 440H705Q698 445 698 453L701 476Q704 500 704 528Q704 558 697 578T678 609T643 625T596 632T532 634H485Q397 633 392 631Q388 629 386 622Q385 619 355 499T324 377Q347 376 372 376H398Q464 376 489 391T534 472Q538 488 540 490T557 493Q562 493 565 493T570 492T572 491T574 487T577 483L544 351Q511 218 508 216Q505 213 492 213Z"></path><path stroke-width="0" id="E2687-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2687-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2687-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2687-MJMAIN-2207" d="M46 676Q46 679 51 683H781Q786 679 786 676Q786 674 617 326T444 -26Q439 -33 416 -33T388 -26Q385 -22 216 326T46 676ZM697 596Q697 597 445 597T193 596Q195 591 319 336T445 80L697 596Z"></path><path stroke-width="0" id="E2687-MJMAIN-6C" d="M42 46H56Q95 46 103 60V68Q103 77 103 91T103 124T104 167T104 217T104 272T104 329Q104 366 104 407T104 482T104 542T103 586T103 603Q100 622 89 628T44 637H26V660Q26 683 28 683L38 684Q48 685 67 686T104 688Q121 689 141 690T171 693T182 694H185V379Q185 62 186 60Q190 52 198 49Q219 46 247 46H263V0H255L232 1Q209 2 183 2T145 3T107 3T57 1L34 0H26V46H42Z"></path><path stroke-width="0" id="E2687-MJMAIN-6E" d="M41 46H55Q94 46 102 60V68Q102 77 102 91T102 122T103 161T103 203Q103 234 103 269T102 328V351Q99 370 88 376T43 385H25V408Q25 431 27 431L37 432Q47 433 65 434T102 436Q119 437 138 438T167 441T178 442H181V402Q181 364 182 364T187 369T199 384T218 402T247 421T285 437Q305 442 336 442Q450 438 463 329Q464 322 464 190V104Q464 66 466 59T477 49Q498 46 526 46H542V0H534L510 1Q487 2 460 2T422 3Q319 3 310 0H302V46H318Q379 46 379 62Q380 64 380 200Q379 335 378 343Q372 371 358 385T334 402T308 404Q263 404 229 370Q202 343 195 315T187 232V168V108Q187 78 188 68T191 55T200 49Q221 46 249 46H265V0H257L234 1Q210 2 183 2T145 3Q42 3 33 0H25V46H41Z"></path><path stroke-width="0" id="E2687-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2687-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2687-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2687-MJMAIN-2223" d="M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z"></path><path stroke-width="0" id="E2687-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2687-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2687-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path><path stroke-width="0" id="E2687-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2687-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2687-MJMATHI-45" x="0" y="0"></use><use xlink:href="#E2687-MJMAIN-5B" x="764" y="0"></use><g transform="translate(1042,0)"><use xlink:href="#E2687-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2687-MJMATHI-74" x="1100" y="-213"></use></g><use xlink:href="#E2687-MJMAIN-2207" x="2175" y="0"></use><g transform="translate(3174,0)"><use xlink:href="#E2687-MJMAIN-6C"></use><use xlink:href="#E2687-MJMAIN-6E" x="278" y="0"></use></g><use xlink:href="#E2687-MJMATHI-3C0" x="4175" y="0"></use><use xlink:href="#E2687-MJMAIN-28" x="4748" y="0"></use><g transform="translate(5137,0)"><use xlink:href="#E2687-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2687-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2687-MJMAIN-2223" x="6520" y="0"></use><g transform="translate(7076,0)"><use xlink:href="#E2687-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2687-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2687-MJMAIN-3B" x="8044" y="0"></use><use xlink:href="#E2687-MJMATHI-3B8" x="8489" y="0"></use><use xlink:href="#E2687-MJMAIN-29" x="8958" y="0"></use><use xlink:href="#E2687-MJMAIN-5D" x="9347" y="0"></use></g></svg></span><script type="math/tex">E[\Psi_t\nabla\ln\pi(A_t \mid S_t;\theta)]</script><span> 为梯度方向进行迭代更新，其中 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="19.789ex" height="2.807ex" viewBox="0 -874.2 8520.3 1208.4" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2688-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2688-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2688-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path stroke-width="0" id="E2688-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2688-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2688-MJMATHI-47" d="M50 252Q50 367 117 473T286 641T490 704Q580 704 633 653Q642 643 648 636T656 626L657 623Q660 623 684 649Q691 655 699 663T715 679T725 690L740 705H746Q760 705 760 698Q760 694 728 561Q692 422 692 421Q690 416 687 415T669 413H653Q647 419 647 422Q647 423 648 429T650 449T651 481Q651 552 619 605T510 659Q492 659 471 656T418 643T357 615T294 567T236 496T189 394T158 260Q156 242 156 221Q156 173 170 136T206 79T256 45T308 28T353 24Q407 24 452 47T514 106Q517 114 529 161T541 214Q541 222 528 224T468 227H431Q425 233 425 235T427 254Q431 267 437 273H454Q494 271 594 271Q634 271 659 271T695 272T707 272Q721 272 721 263Q721 261 719 249Q714 230 709 228Q706 227 694 227Q674 227 653 224Q646 221 643 215T629 164Q620 131 614 108Q589 6 586 3Q584 1 581 1Q571 1 553 21T530 52Q530 53 528 52T522 47Q448 -22 322 -22Q201 -22 126 55T50 252Z"></path><path stroke-width="0" id="E2688-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2688-MJMATHI-42" d="M231 637Q204 637 199 638T194 649Q194 676 205 682Q206 683 335 683Q594 683 608 681Q671 671 713 636T756 544Q756 480 698 429T565 360L555 357Q619 348 660 311T702 219Q702 146 630 78T453 1Q446 0 242 0Q42 0 39 2Q35 5 35 10Q35 17 37 24Q42 43 47 45Q51 46 62 46H68Q95 46 128 49Q142 52 147 61Q150 65 219 339T288 628Q288 635 231 637ZM649 544Q649 574 634 600T585 634Q578 636 493 637Q473 637 451 637T416 636H403Q388 635 384 626Q382 622 352 506Q352 503 351 500L320 374H401Q482 374 494 376Q554 386 601 434T649 544ZM595 229Q595 273 572 302T512 336Q506 337 429 337Q311 337 310 336Q310 334 293 263T258 122L240 52Q240 48 252 48T333 46Q422 46 429 47Q491 54 543 105T595 229Z"></path><path stroke-width="0" id="E2688-MJMATHI-73" d="M131 289Q131 321 147 354T203 415T300 442Q362 442 390 415T419 355Q419 323 402 308T364 292Q351 292 340 300T328 326Q328 342 337 354T354 372T367 378Q368 378 368 379Q368 382 361 388T336 399T297 405Q249 405 227 379T204 326Q204 301 223 291T278 274T330 259Q396 230 396 163Q396 135 385 107T352 51T289 7T195 -10Q118 -10 86 19T53 87Q53 126 74 143T118 160Q133 160 146 151T160 120Q160 94 142 76T111 58Q109 57 108 57T107 55Q108 52 115 47T146 34T201 27Q237 27 263 38T301 66T318 97T323 122Q323 150 302 164T254 181T195 196T148 231Q131 256 131 289Z"></path><path stroke-width="0" id="E2688-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2688-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2688-MJMATHI-74" x="1100" y="-213"></use><use xlink:href="#E2688-MJMAIN-3D" x="1411" y="0"></use><g transform="translate(2466,0)"><use xlink:href="#E2688-MJMATHI-3B3" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2688-MJMATHI-74" x="778" y="513"></use></g><use xlink:href="#E2688-MJMAIN-28" x="3372" y="0"></use><g transform="translate(3761,0)"><use xlink:href="#E2688-MJMATHI-47" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2688-MJMATHI-74" x="1111" y="-213"></use></g><use xlink:href="#E2688-MJMAIN-2212" x="5125" y="0"></use><use xlink:href="#E2688-MJMATHI-42" x="6125" y="0"></use><use xlink:href="#E2688-MJMAIN-28" x="6884" y="0"></use><use xlink:href="#E2688-MJMATHI-73" x="7273" y="0"></use><use xlink:href="#E2688-MJMAIN-29" x="7742" y="0"></use><use xlink:href="#E2688-MJMAIN-29" x="8131" y="0"></use></g></svg></span><script type="math/tex">\Psi_t = \gamma^t(G_t-B(s))</script><span> 。J. Schulman 等人在文章《High-dimensional continuous control using generalized advantage estimation》中指出，</span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="2.632ex" height="2.228ex" viewBox="0 -749.6 1133.3 959.2" role="img" focusable="false" style="vertical-align: -0.487ex;"><defs><path stroke-width="0" id="E2689-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2689-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2689-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2689-MJMATHI-74" x="1100" y="-213"></use></g></svg></span><script type="math/tex">\Psi_t</script><span> 并不拘泥于以上形式，还可以为以下几种形式：</span></p><ul><li><span>（动作价值）</span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="17.698ex" height="2.807ex" viewBox="0 -874.2 7620 1208.4" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2690-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2690-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2690-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path stroke-width="0" id="E2690-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2690-MJMATHI-71" d="M33 157Q33 258 109 349T280 441Q340 441 372 389Q373 390 377 395T388 406T404 418Q438 442 450 442Q454 442 457 439T460 434Q460 425 391 149Q320 -135 320 -139Q320 -147 365 -148H390Q396 -156 396 -157T393 -175Q389 -188 383 -194H370Q339 -192 262 -192Q234 -192 211 -192T174 -192T157 -193Q143 -193 143 -185Q143 -182 145 -170Q149 -154 152 -151T172 -148Q220 -148 230 -141Q238 -136 258 -53T279 32Q279 33 272 29Q224 -10 172 -10Q117 -10 75 30T33 157ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"></path><path stroke-width="0" id="E2690-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2690-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2690-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2690-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2690-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2690-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2690-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2690-MJMATHI-74" x="1100" y="-213"></use><use xlink:href="#E2690-MJMAIN-3D" x="1411" y="0"></use><g transform="translate(2466,0)"><use xlink:href="#E2690-MJMATHI-3B3" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2690-MJMATHI-74" x="778" y="513"></use></g><g transform="translate(3372,0)"><use xlink:href="#E2690-MJMATHI-71" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2690-MJMATHI-3C0" x="630" y="-213"></use></g><use xlink:href="#E2690-MJMAIN-28" x="4323" y="0"></use><g transform="translate(4712,0)"><use xlink:href="#E2690-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2690-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2690-MJMAIN-2C" x="5681" y="0"></use><g transform="translate(6125,0)"><use xlink:href="#E2690-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2690-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2690-MJMAIN-29" x="7231" y="0"></use></g></svg></span><script type="math/tex">\Psi_t = \gamma^tq_\pi(S_t,A_t)</script><span> ；</span></li><li><span>（优势函数）</span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="28.184ex" height="2.807ex" viewBox="0 -874.2 12134.9 1208.4" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2691-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2691-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2691-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path stroke-width="0" id="E2691-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2691-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2691-MJMATHI-71" d="M33 157Q33 258 109 349T280 441Q340 441 372 389Q373 390 377 395T388 406T404 418Q438 442 450 442Q454 442 457 439T460 434Q460 425 391 149Q320 -135 320 -139Q320 -147 365 -148H390Q396 -156 396 -157T393 -175Q389 -188 383 -194H370Q339 -192 262 -192Q234 -192 211 -192T174 -192T157 -193Q143 -193 143 -185Q143 -182 145 -170Q149 -154 152 -151T172 -148Q220 -148 230 -141Q238 -136 258 -53T279 32Q279 33 272 29Q224 -10 172 -10Q117 -10 75 30T33 157ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"></path><path stroke-width="0" id="E2691-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2691-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2691-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2691-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2691-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2691-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2691-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2691-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2691-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2691-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2691-MJMATHI-74" x="1100" y="-213"></use><use xlink:href="#E2691-MJMAIN-3D" x="1411" y="0"></use><g transform="translate(2466,0)"><use xlink:href="#E2691-MJMATHI-3B3" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2691-MJMATHI-74" x="778" y="513"></use></g><use xlink:href="#E2691-MJMAIN-5B" x="3372" y="0"></use><g transform="translate(3650,0)"><use xlink:href="#E2691-MJMATHI-71" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2691-MJMATHI-3C0" x="630" y="-213"></use></g><use xlink:href="#E2691-MJMAIN-28" x="4601" y="0"></use><g transform="translate(4990,0)"><use xlink:href="#E2691-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2691-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2691-MJMAIN-2C" x="5959" y="0"></use><g transform="translate(6403,0)"><use xlink:href="#E2691-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2691-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2691-MJMAIN-29" x="7509" y="0"></use><use xlink:href="#E2691-MJMAIN-2212" x="8120" y="0"></use><g transform="translate(9120,0)"><use xlink:href="#E2691-MJMATHI-76" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2691-MJMATHI-3C0" x="685" y="-213"></use></g><use xlink:href="#E2691-MJMAIN-28" x="10110" y="0"></use><g transform="translate(10499,0)"><use xlink:href="#E2691-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2691-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2691-MJMAIN-29" x="11467" y="0"></use><use xlink:href="#E2691-MJMAIN-5D" x="11856" y="0"></use></g></svg></span><script type="math/tex">\Psi_t = \gamma^t[q_\pi(S_t,A_t)-v_\pi(S_t)]</script><span> ；</span></li><li><span>（时序差分）</span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="35.561ex" height="2.807ex" viewBox="0 -874.2 15311 1208.4" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2692-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2692-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2692-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path stroke-width="0" id="E2692-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2692-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2692-MJMATHI-52" d="M230 637Q203 637 198 638T193 649Q193 676 204 682Q206 683 378 683Q550 682 564 680Q620 672 658 652T712 606T733 563T739 529Q739 484 710 445T643 385T576 351T538 338L545 333Q612 295 612 223Q612 212 607 162T602 80V71Q602 53 603 43T614 25T640 16Q668 16 686 38T712 85Q717 99 720 102T735 105Q755 105 755 93Q755 75 731 36Q693 -21 641 -21H632Q571 -21 531 4T487 82Q487 109 502 166T517 239Q517 290 474 313Q459 320 449 321T378 323H309L277 193Q244 61 244 59Q244 55 245 54T252 50T269 48T302 46H333Q339 38 339 37T336 19Q332 6 326 0H311Q275 2 180 2Q146 2 117 2T71 2T50 1Q33 1 33 10Q33 12 36 24Q41 43 46 45Q50 46 61 46H67Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628Q287 635 230 637ZM630 554Q630 586 609 608T523 636Q521 636 500 636T462 637H440Q393 637 386 627Q385 624 352 494T319 361Q319 360 388 360Q466 361 492 367Q556 377 592 426Q608 449 619 486T630 554Z"></path><path stroke-width="0" id="E2692-MJMAIN-2B" d="M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"></path><path stroke-width="0" id="E2692-MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="0" id="E2692-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2692-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2692-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2692-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2692-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2692-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2692-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2692-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2692-MJMATHI-74" x="1100" y="-213"></use><use xlink:href="#E2692-MJMAIN-3D" x="1411" y="0"></use><g transform="translate(2466,0)"><use xlink:href="#E2692-MJMATHI-3B3" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2692-MJMATHI-74" x="778" y="513"></use></g><use xlink:href="#E2692-MJMAIN-5B" x="3372" y="0"></use><g transform="translate(3650,0)"><use xlink:href="#E2692-MJMATHI-52" x="0" y="0"></use><g transform="translate(759,-150)"><use transform="scale(0.707)" xlink:href="#E2692-MJMATHI-74" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2692-MJMAIN-2B" x="361" y="0"></use><use transform="scale(0.707)" xlink:href="#E2692-MJMAIN-31" x="1139" y="0"></use></g></g><use xlink:href="#E2692-MJMAIN-2B" x="5890" y="0"></use><use xlink:href="#E2692-MJMATHI-3B3" x="6891" y="0"></use><g transform="translate(7434,0)"><use xlink:href="#E2692-MJMATHI-76" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2692-MJMATHI-3C0" x="685" y="-213"></use></g><use xlink:href="#E2692-MJMAIN-28" x="8424" y="0"></use><g transform="translate(8813,0)"><use xlink:href="#E2692-MJMATHI-53" x="0" y="0"></use><g transform="translate(613,-150)"><use transform="scale(0.707)" xlink:href="#E2692-MJMATHI-74" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2692-MJMAIN-2B" x="361" y="0"></use><use transform="scale(0.707)" xlink:href="#E2692-MJMAIN-31" x="1139" y="0"></use></g></g><use xlink:href="#E2692-MJMAIN-29" x="10685" y="0"></use><use xlink:href="#E2692-MJMAIN-2212" x="11296" y="0"></use><g transform="translate(12296,0)"><use xlink:href="#E2692-MJMATHI-76" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2692-MJMATHI-3C0" x="685" y="-213"></use></g><use xlink:href="#E2692-MJMAIN-28" x="13286" y="0"></use><g transform="translate(13675,0)"><use xlink:href="#E2692-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2692-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2692-MJMAIN-29" x="14644" y="0"></use><use xlink:href="#E2692-MJMAIN-5D" x="15033" y="0"></use></g></svg></span><script type="math/tex">\Psi_t = \gamma^t[R_{t+1} + \gamma v_\pi(S_{t+1})-v_\pi(S_t)]</script><span> ；</span></li></ul><p><span>其中，前两者是使用价值函数 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="9.865ex" height="2.71ex" viewBox="0 -832.7 4247.4 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2693-MJMATHI-71" d="M33 157Q33 258 109 349T280 441Q340 441 372 389Q373 390 377 395T388 406T404 418Q438 442 450 442Q454 442 457 439T460 434Q460 425 391 149Q320 -135 320 -139Q320 -147 365 -148H390Q396 -156 396 -157T393 -175Q389 -188 383 -194H370Q339 -192 262 -192Q234 -192 211 -192T174 -192T157 -193Q143 -193 143 -185Q143 -182 145 -170Q149 -154 152 -151T172 -148Q220 -148 230 -141Q238 -136 258 -53T279 32Q279 33 272 29Q224 -10 172 -10Q117 -10 75 30T33 157ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"></path><path stroke-width="0" id="E2693-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2693-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2693-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2693-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2693-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2693-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2693-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2693-MJMATHI-71" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2693-MJMATHI-3C0" x="630" y="-213"></use><use xlink:href="#E2693-MJMAIN-28" x="951" y="0"></use><g transform="translate(1340,0)"><use xlink:href="#E2693-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2693-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2693-MJMAIN-2C" x="2308" y="0"></use><g transform="translate(2753,0)"><use xlink:href="#E2693-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2693-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2693-MJMAIN-29" x="3858" y="0"></use></g></svg></span><script type="math/tex">q_\pi(S_t,A_t)</script><span> 来估计回报的，而时序差分则是用 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="17.242ex" height="2.71ex" viewBox="0 -832.7 7423.5 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2694-MJMATHI-52" d="M230 637Q203 637 198 638T193 649Q193 676 204 682Q206 683 378 683Q550 682 564 680Q620 672 658 652T712 606T733 563T739 529Q739 484 710 445T643 385T576 351T538 338L545 333Q612 295 612 223Q612 212 607 162T602 80V71Q602 53 603 43T614 25T640 16Q668 16 686 38T712 85Q717 99 720 102T735 105Q755 105 755 93Q755 75 731 36Q693 -21 641 -21H632Q571 -21 531 4T487 82Q487 109 502 166T517 239Q517 290 474 313Q459 320 449 321T378 323H309L277 193Q244 61 244 59Q244 55 245 54T252 50T269 48T302 46H333Q339 38 339 37T336 19Q332 6 326 0H311Q275 2 180 2Q146 2 117 2T71 2T50 1Q33 1 33 10Q33 12 36 24Q41 43 46 45Q50 46 61 46H67Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628Q287 635 230 637ZM630 554Q630 586 609 608T523 636Q521 636 500 636T462 637H440Q393 637 386 627Q385 624 352 494T319 361Q319 360 388 360Q466 361 492 367Q556 377 592 426Q608 449 619 486T630 554Z"></path><path stroke-width="0" id="E2694-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2694-MJMAIN-2B" d="M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"></path><path stroke-width="0" id="E2694-MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="0" id="E2694-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2694-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2694-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2694-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2694-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2694-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2694-MJMATHI-52" x="0" y="0"></use><g transform="translate(759,-150)"><use transform="scale(0.707)" xlink:href="#E2694-MJMATHI-74" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2694-MJMAIN-2B" x="361" y="0"></use><use transform="scale(0.707)" xlink:href="#E2694-MJMAIN-31" x="1139" y="0"></use></g><use xlink:href="#E2694-MJMAIN-2B" x="2240" y="0"></use><use xlink:href="#E2694-MJMATHI-3B3" x="3240" y="0"></use><g transform="translate(3783,0)"><use xlink:href="#E2694-MJMATHI-76" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2694-MJMATHI-3C0" x="685" y="-213"></use></g><use xlink:href="#E2694-MJMAIN-28" x="4773" y="0"></use><g transform="translate(5162,0)"><use xlink:href="#E2694-MJMATHI-53" x="0" y="0"></use><g transform="translate(613,-150)"><use transform="scale(0.707)" xlink:href="#E2694-MJMATHI-74" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2694-MJMAIN-2B" x="361" y="0"></use><use transform="scale(0.707)" xlink:href="#E2694-MJMAIN-31" x="1139" y="0"></use></g></g><use xlink:href="#E2694-MJMAIN-29" x="7034" y="0"></use></g></svg></span><script type="math/tex">R_{t+1}+\gamma v_\pi(S_{t+1})</script><span> 代表回报，再减去基线函数 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="12.952ex" height="2.71ex" viewBox="0 -832.7 5576.7 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2695-MJMATHI-42" d="M231 637Q204 637 199 638T194 649Q194 676 205 682Q206 683 335 683Q594 683 608 681Q671 671 713 636T756 544Q756 480 698 429T565 360L555 357Q619 348 660 311T702 219Q702 146 630 78T453 1Q446 0 242 0Q42 0 39 2Q35 5 35 10Q35 17 37 24Q42 43 47 45Q51 46 62 46H68Q95 46 128 49Q142 52 147 61Q150 65 219 339T288 628Q288 635 231 637ZM649 544Q649 574 634 600T585 634Q578 636 493 637Q473 637 451 637T416 636H403Q388 635 384 626Q382 622 352 506Q352 503 351 500L320 374H401Q482 374 494 376Q554 386 601 434T649 544ZM595 229Q595 273 572 302T512 336Q506 337 429 337Q311 337 310 336Q310 334 293 263T258 122L240 52Q240 48 252 48T333 46Q422 46 429 47Q491 54 543 105T595 229Z"></path><path stroke-width="0" id="E2695-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2695-MJMATHI-73" d="M131 289Q131 321 147 354T203 415T300 442Q362 442 390 415T419 355Q419 323 402 308T364 292Q351 292 340 300T328 326Q328 342 337 354T354 372T367 378Q368 378 368 379Q368 382 361 388T336 399T297 405Q249 405 227 379T204 326Q204 301 223 291T278 274T330 259Q396 230 396 163Q396 135 385 107T352 51T289 7T195 -10Q118 -10 86 19T53 87Q53 126 74 143T118 160Q133 160 146 151T160 120Q160 94 142 76T111 58Q109 57 108 57T107 55Q108 52 115 47T146 34T201 27Q237 27 263 38T301 66T318 97T323 122Q323 150 302 164T254 181T195 196T148 231Q131 256 131 289Z"></path><path stroke-width="0" id="E2695-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2695-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path stroke-width="0" id="E2695-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2695-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2695-MJMATHI-42" x="0" y="0"></use><use xlink:href="#E2695-MJMAIN-28" x="759" y="0"></use><use xlink:href="#E2695-MJMATHI-73" x="1148" y="0"></use><use xlink:href="#E2695-MJMAIN-29" x="1617" y="0"></use><use xlink:href="#E2695-MJMAIN-3D" x="2283" y="0"></use><g transform="translate(3339,0)"><use xlink:href="#E2695-MJMATHI-76" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2695-MJMATHI-3C0" x="685" y="-213"></use></g><use xlink:href="#E2695-MJMAIN-28" x="4329" y="0"></use><use xlink:href="#E2695-MJMATHI-73" x="4718" y="0"></use><use xlink:href="#E2695-MJMAIN-29" x="5187" y="0"></use></g></svg></span><script type="math/tex">B(s)=v_\pi(s)</script><span> 以减小学习过程中的方差。但由于实际使用时的真实价值函数是不知道的，所以采用自益的方法用价值估计 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="2.411ex" height="2.228ex" viewBox="0 -749.6 1038.3 959.2" role="img" focusable="false" style="vertical-align: -0.487ex;"><defs><path stroke-width="0" id="E2696-MJMATHI-55" d="M107 637Q73 637 71 641Q70 643 70 649Q70 673 81 682Q83 683 98 683Q139 681 234 681Q268 681 297 681T342 682T362 682Q378 682 378 672Q378 670 376 658Q371 641 366 638H364Q362 638 359 638T352 638T343 637T334 637Q295 636 284 634T266 623Q265 621 238 518T184 302T154 169Q152 155 152 140Q152 86 183 55T269 24Q336 24 403 69T501 205L552 406Q599 598 599 606Q599 633 535 637Q511 637 511 648Q511 650 513 660Q517 676 519 679T529 683Q532 683 561 682T645 680Q696 680 723 681T752 682Q767 682 767 672Q767 650 759 642Q756 637 737 637Q666 633 648 597Q646 592 598 404Q557 235 548 205Q515 105 433 42T263 -22Q171 -22 116 34T60 167V183Q60 201 115 421Q164 622 164 628Q164 635 107 637Z"></path><path stroke-width="0" id="E2696-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2696-MJMATHI-55" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2696-MJMATHI-74" x="965" y="-213"></use></g></svg></span><script type="math/tex">U_t</script><span> 作为回报，即 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="41.487ex" height="2.807ex" viewBox="0 -874.2 17862.4 1208.4" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2697-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2697-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2697-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path stroke-width="0" id="E2697-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2697-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2697-MJMATHI-52" d="M230 637Q203 637 198 638T193 649Q193 676 204 682Q206 683 378 683Q550 682 564 680Q620 672 658 652T712 606T733 563T739 529Q739 484 710 445T643 385T576 351T538 338L545 333Q612 295 612 223Q612 212 607 162T602 80V71Q602 53 603 43T614 25T640 16Q668 16 686 38T712 85Q717 99 720 102T735 105Q755 105 755 93Q755 75 731 36Q693 -21 641 -21H632Q571 -21 531 4T487 82Q487 109 502 166T517 239Q517 290 474 313Q459 320 449 321T378 323H309L277 193Q244 61 244 59Q244 55 245 54T252 50T269 48T302 46H333Q339 38 339 37T336 19Q332 6 326 0H311Q275 2 180 2Q146 2 117 2T71 2T50 1Q33 1 33 10Q33 12 36 24Q41 43 46 45Q50 46 61 46H67Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628Q287 635 230 637ZM630 554Q630 586 609 608T523 636Q521 636 500 636T462 637H440Q393 637 386 627Q385 624 352 494T319 361Q319 360 388 360Q466 361 492 367Q556 377 592 426Q608 449 619 486T630 554Z"></path><path stroke-width="0" id="E2697-MJMAIN-2B" d="M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"></path><path stroke-width="0" id="E2697-MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="0" id="E2697-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2697-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2697-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2697-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2697-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2697-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2697-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2697-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2697-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2697-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2697-MJMATHI-74" x="1100" y="-213"></use><use xlink:href="#E2697-MJMAIN-3D" x="1411" y="0"></use><g transform="translate(2466,0)"><use xlink:href="#E2697-MJMATHI-3B3" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2697-MJMATHI-74" x="778" y="513"></use></g><use xlink:href="#E2697-MJMAIN-5B" x="3372" y="0"></use><g transform="translate(3650,0)"><use xlink:href="#E2697-MJMATHI-52" x="0" y="0"></use><g transform="translate(759,-150)"><use transform="scale(0.707)" xlink:href="#E2697-MJMATHI-74" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2697-MJMAIN-2B" x="361" y="0"></use><use transform="scale(0.707)" xlink:href="#E2697-MJMAIN-31" x="1139" y="0"></use></g></g><use xlink:href="#E2697-MJMAIN-2B" x="5890" y="0"></use><use xlink:href="#E2697-MJMATHI-3B3" x="6891" y="0"></use><g transform="translate(7434,0)"><use xlink:href="#E2697-MJMATHI-76" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2697-MJMATHI-3C0" x="685" y="-213"></use></g><use xlink:href="#E2697-MJMAIN-28" x="8424" y="0"></use><g transform="translate(8813,0)"><use xlink:href="#E2697-MJMATHI-53" x="0" y="0"></use><g transform="translate(613,-150)"><use transform="scale(0.707)" xlink:href="#E2697-MJMATHI-74" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2697-MJMAIN-2B" x="361" y="0"></use><use transform="scale(0.707)" xlink:href="#E2697-MJMAIN-31" x="1139" y="0"></use></g></g><use xlink:href="#E2697-MJMAIN-3B" x="10685" y="0"></use><use xlink:href="#E2697-MJMAINB-77" x="11129" y="0"></use><use xlink:href="#E2697-MJMAIN-29" x="11960" y="0"></use><use xlink:href="#E2697-MJMAIN-2212" x="12572" y="0"></use><g transform="translate(13572,0)"><use xlink:href="#E2697-MJMATHI-76" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2697-MJMATHI-3C0" x="685" y="-213"></use></g><use xlink:href="#E2697-MJMAIN-28" x="14562" y="0"></use><g transform="translate(14951,0)"><use xlink:href="#E2697-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2697-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2697-MJMAIN-3B" x="15919" y="0"></use><use xlink:href="#E2697-MJMAINB-77" x="16364" y="0"></use><use xlink:href="#E2697-MJMAIN-29" x="17195" y="0"></use><use xlink:href="#E2697-MJMAIN-5D" x="17584" y="0"></use></g></svg></span><script type="math/tex">\Psi_t = \gamma^t[R_{t+1} + \gamma v_\pi(S_{t+1};\bold w)-v_\pi(S_t;\bold w)]</script><span> ，这里估计值 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="4.864ex" height="2.71ex" viewBox="0 -832.7 2094 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2707-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2707-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2707-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2707-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2707-MJMATHI-76" x="0" y="0"></use><use xlink:href="#E2707-MJMAIN-28" x="485" y="0"></use><use xlink:href="#E2707-MJMAINB-77" x="874" y="0"></use><use xlink:href="#E2707-MJMAIN-29" x="1705" y="0"></use></g></svg></span><script type="math/tex">v(\bold w)</script><span> 就是评论者。另外，只有采用了自益的方法，即用价值估计来估计回报，并引入了偏差，才是执行者 / 评论者算法；用价值估计来做基线并没有带来偏差，所以带基线的简单策略梯度算法不属于执行者 / 评论者算法。</span></p><p><span>当使用 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="11.687ex" height="2.71ex" viewBox="0 -832.7 5031.9 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2701-MJMATHI-71" d="M33 157Q33 258 109 349T280 441Q340 441 372 389Q373 390 377 395T388 406T404 418Q438 442 450 442Q454 442 457 439T460 434Q460 425 391 149Q320 -135 320 -139Q320 -147 365 -148H390Q396 -156 396 -157T393 -175Q389 -188 383 -194H370Q339 -192 262 -192Q234 -192 211 -192T174 -192T157 -193Q143 -193 143 -185Q143 -182 145 -170Q149 -154 152 -151T172 -148Q220 -148 230 -141Q238 -136 258 -53T279 32Q279 33 272 29Q224 -10 172 -10Q117 -10 75 30T33 157ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"></path><path stroke-width="0" id="E2701-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2701-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2701-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2701-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2701-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2701-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2701-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2701-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2701-MJMATHI-71" x="0" y="0"></use><use xlink:href="#E2701-MJMAIN-28" x="460" y="0"></use><g transform="translate(849,0)"><use xlink:href="#E2701-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2701-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2701-MJMAIN-2C" x="1817" y="0"></use><g transform="translate(2261,0)"><use xlink:href="#E2701-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2701-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2701-MJMAIN-3B" x="3367" y="0"></use><use xlink:href="#E2701-MJMAINB-77" x="3811" y="0"></use><use xlink:href="#E2701-MJMAIN-29" x="4642" y="0"></use></g></svg></span><script type="math/tex">q(S_t,A_t;\bold w)</script><span> 作为回报估计，并取 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="19.52ex" height="2.807ex" viewBox="0 -874.2 8404.5 1208.4" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2700-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2700-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2700-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path stroke-width="0" id="E2700-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2700-MJMATHI-71" d="M33 157Q33 258 109 349T280 441Q340 441 372 389Q373 390 377 395T388 406T404 418Q438 442 450 442Q454 442 457 439T460 434Q460 425 391 149Q320 -135 320 -139Q320 -147 365 -148H390Q396 -156 396 -157T393 -175Q389 -188 383 -194H370Q339 -192 262 -192Q234 -192 211 -192T174 -192T157 -193Q143 -193 143 -185Q143 -182 145 -170Q149 -154 152 -151T172 -148Q220 -148 230 -141Q238 -136 258 -53T279 32Q279 33 272 29Q224 -10 172 -10Q117 -10 75 30T33 157ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"></path><path stroke-width="0" id="E2700-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2700-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2700-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2700-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2700-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2700-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2700-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2700-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2700-MJMATHI-74" x="1100" y="-213"></use><use xlink:href="#E2700-MJMAIN-3D" x="1411" y="0"></use><g transform="translate(2466,0)"><use xlink:href="#E2700-MJMATHI-3B3" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2700-MJMATHI-74" x="778" y="513"></use></g><use xlink:href="#E2700-MJMATHI-71" x="3372" y="0"></use><use xlink:href="#E2700-MJMAIN-28" x="3832" y="0"></use><g transform="translate(4221,0)"><use xlink:href="#E2700-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2700-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2700-MJMAIN-2C" x="5189" y="0"></use><g transform="translate(5634,0)"><use xlink:href="#E2700-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2700-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2700-MJMAIN-3B" x="6739" y="0"></use><use xlink:href="#E2700-MJMAINB-77" x="7184" y="0"></use><use xlink:href="#E2700-MJMAIN-29" x="8015" y="0"></use></g></svg></span><script type="math/tex">\Psi_t = \gamma^tq(S_t,A_t;\bold w)</script><span> 时的同策算法称为动作价值执行者 / 评论者算法：</span></p><div contenteditable="false" spellcheck="false" class="mathjax-block md-end-block md-math-block md-rawblock" id="mathjax-n14" cid="n14" mdtype="math_block"><div class="md-rawblock-container md-math-container" contenteditable="false" tabindex="-1">
						<div class="MathJax_SVG_Display"><span class="MathJax_SVG" id="MathJax-Element-1984-Frame" tabindex="-1" style="font-size: 100%; display: inline-block; zoom: 0.971345;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="101.294ex" height="71.295ex" viewBox="-18.1 -43.5 43612.5 30696.5" role="img" focusable="false" style="vertical-align: -71.194ex; margin-left: -0.042ex; max-width: 100%;"><defs><path stroke-width="0" id="E2681-MJMAINB-38" d="M80 474Q80 561 139 607T278 654Q357 654 411 632Q490 593 494 509Q494 424 416 376L407 371L418 364Q432 356 447 345T481 312T513 260T526 192Q526 100 461 45T285 -11Q184 -11 116 32T48 164Q48 181 50 196T58 225T69 249T84 270T100 286T117 300T134 311T149 321T162 329L152 336Q120 360 100 397T80 474ZM347 404Q404 446 404 503Q404 579 317 599Q309 600 276 600Q178 600 170 538Q170 532 171 527T173 518T178 509T184 501T194 492T205 484T219 476T235 467T254 456T275 445L347 404ZM289 47Q323 47 351 54T402 82T425 137Q425 147 421 161Q411 183 391 197T303 249Q224 293 223 293Q220 291 215 288T197 273T175 248T157 213T149 167Q149 109 188 78T289 47Z"></path><path stroke-width="0" id="E2681-MJMAINB-2D" d="M13 166V278H318V166H13Z"></path><path stroke-width="0" id="E2681-MJMAINB-31" d="M481 0L294 3Q136 3 109 0H96V62H227V304Q227 546 225 546Q169 529 97 529H80V591H97Q231 591 308 647L319 655H333Q355 655 359 644Q361 640 361 351V62H494V0H481Z"></path><path stroke-width="0" id="E2681-MJMAINB-2F" d="M451 730Q460 750 479 750Q492 750 501 740T510 718Q508 708 318 244L122 -232Q112 -250 95 -250Q82 -250 73 -241T64 -218Q66 -205 258 261T451 730Z"></path><path stroke-width="0" id="E2681-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2681-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2681-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path><path stroke-width="0" id="E2681-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2681-MJMATHI-3B1" d="M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z"></path><path stroke-width="0" id="E2681-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2681-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2681-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2681-MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="0" id="E2681-MJMAIN-2E" d="M78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60Z"></path><path stroke-width="0" id="E2681-MJMAIN-2190" d="M944 261T944 250T929 230H165Q167 228 182 216T211 189T244 152T277 96T303 25Q308 7 308 0Q308 -11 288 -11Q281 -11 278 -11T272 -7T267 2T263 21Q245 94 195 151T73 236Q58 242 55 247Q55 254 59 257T73 264Q121 283 158 314T215 375T247 434T264 480L267 497Q269 503 270 505T275 509T288 511Q308 511 308 500Q308 493 303 475Q293 438 278 406T246 352T215 315T185 287T165 270H929Q944 261 944 250Z"></path><path stroke-width="0" id="E2681-MJMAIN-32" d="M109 429Q82 429 66 447T50 491Q50 562 103 614T235 666Q326 666 387 610T449 465Q449 422 429 383T381 315T301 241Q265 210 201 149L142 93L218 92Q375 92 385 97Q392 99 409 186V189H449V186Q448 183 436 95T421 3V0H50V19V31Q50 38 56 46T86 81Q115 113 136 137Q145 147 170 174T204 211T233 244T261 278T284 308T305 340T320 369T333 401T340 431T343 464Q343 527 309 573T212 619Q179 619 154 602T119 569T109 550Q109 549 114 549Q132 549 151 535T170 489Q170 464 154 447T109 429Z"></path><path stroke-width="0" id="E2681-MJMATHI-49" d="M43 1Q26 1 26 10Q26 12 29 24Q34 43 39 45Q42 46 54 46H60Q120 46 136 53Q137 53 138 54Q143 56 149 77T198 273Q210 318 216 344Q286 624 286 626Q284 630 284 631Q274 637 213 637H193Q184 643 189 662Q193 677 195 680T209 683H213Q285 681 359 681Q481 681 487 683H497Q504 676 504 672T501 655T494 639Q491 637 471 637Q440 637 407 634Q393 631 388 623Q381 609 337 432Q326 385 315 341Q245 65 245 59Q245 52 255 50T307 46H339Q345 38 345 37T342 19Q338 6 332 0H316Q279 2 179 2Q143 2 113 2T65 2T43 1Z"></path><path stroke-width="0" id="E2681-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2681-MJMAIN-22C5" d="M78 250Q78 274 95 292T138 310Q162 310 180 294T199 251Q199 226 182 208T139 190T96 207T78 250Z"></path><path stroke-width="0" id="E2681-MJMAIN-2223" d="M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z"></path><path stroke-width="0" id="E2681-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2681-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2681-MJMAIN-33" d="M127 463Q100 463 85 480T69 524Q69 579 117 622T233 665Q268 665 277 664Q351 652 390 611T430 522Q430 470 396 421T302 350L299 348Q299 347 308 345T337 336T375 315Q457 262 457 175Q457 96 395 37T238 -22Q158 -22 100 21T42 130Q42 158 60 175T105 193Q133 193 151 175T169 130Q169 119 166 110T159 94T148 82T136 74T126 70T118 67L114 66Q165 21 238 21Q293 21 321 74Q338 107 338 175V195Q338 290 274 322Q259 328 213 329L171 330L168 332Q166 335 166 348Q166 366 174 366Q202 366 232 371Q266 376 294 413T322 525V533Q322 590 287 612Q265 626 240 626Q208 626 181 615T143 592T132 580H135Q138 579 143 578T153 573T165 566T175 555T183 540T186 520Q186 498 172 481T127 463Z"></path><path stroke-width="0" id="E2681-MJMATHI-52" d="M230 637Q203 637 198 638T193 649Q193 676 204 682Q206 683 378 683Q550 682 564 680Q620 672 658 652T712 606T733 563T739 529Q739 484 710 445T643 385T576 351T538 338L545 333Q612 295 612 223Q612 212 607 162T602 80V71Q602 53 603 43T614 25T640 16Q668 16 686 38T712 85Q717 99 720 102T735 105Q755 105 755 93Q755 75 731 36Q693 -21 641 -21H632Q571 -21 531 4T487 82Q487 109 502 166T517 239Q517 290 474 313Q459 320 449 321T378 323H309L277 193Q244 61 244 59Q244 55 245 54T252 50T269 48T302 46H333Q339 38 339 37T336 19Q332 6 326 0H311Q275 2 180 2Q146 2 117 2T71 2T50 1Q33 1 33 10Q33 12 36 24Q41 43 46 45Q50 46 61 46H67Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628Q287 635 230 637ZM630 554Q630 586 609 608T523 636Q521 636 500 636T462 637H440Q393 637 386 627Q385 624 352 494T319 361Q319 360 388 360Q466 361 492 367Q556 377 592 426Q608 449 619 486T630 554Z"></path><path stroke-width="0" id="E2681-MJMAIN-2032" d="M79 43Q73 43 52 49T30 61Q30 68 85 293T146 528Q161 560 198 560Q218 560 240 545T262 501Q262 496 260 486Q259 479 173 263T84 45T79 43Z"></path><path stroke-width="0" id="E2681-MJMATHI-55" d="M107 637Q73 637 71 641Q70 643 70 649Q70 673 81 682Q83 683 98 683Q139 681 234 681Q268 681 297 681T342 682T362 682Q378 682 378 672Q378 670 376 658Q371 641 366 638H364Q362 638 359 638T352 638T343 637T334 637Q295 636 284 634T266 623Q265 621 238 518T184 302T154 169Q152 155 152 140Q152 86 183 55T269 24Q336 24 403 69T501 205L552 406Q599 598 599 606Q599 633 535 637Q511 637 511 648Q511 650 513 660Q517 676 519 679T529 683Q532 683 561 682T645 680Q696 680 723 681T752 682Q767 682 767 672Q767 650 759 642Q756 637 737 637Q666 633 648 597Q646 592 598 404Q557 235 548 205Q515 105 433 42T263 -22Q171 -22 116 34T60 167V183Q60 201 115 421Q164 622 164 628Q164 635 107 637Z"></path><path stroke-width="0" id="E2681-MJMAIN-2B" d="M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"></path><path stroke-width="0" id="E2681-MJMATHI-71" d="M33 157Q33 258 109 349T280 441Q340 441 372 389Q373 390 377 395T388 406T404 418Q438 442 450 442Q454 442 457 439T460 434Q460 425 391 149Q320 -135 320 -139Q320 -147 365 -148H390Q396 -156 396 -157T393 -175Q389 -188 383 -194H370Q339 -192 262 -192Q234 -192 211 -192T174 -192T157 -193Q143 -193 143 -185Q143 -182 145 -170Q149 -154 152 -151T172 -148Q220 -148 230 -141Q238 -136 258 -53T279 32Q279 33 272 29Q224 -10 172 -10Q117 -10 75 30T33 157ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"></path><path stroke-width="0" id="E2681-MJMAIN-34" d="M462 0Q444 3 333 3Q217 3 199 0H190V46H221Q241 46 248 46T265 48T279 53T286 61Q287 63 287 115V165H28V211L179 442Q332 674 334 675Q336 677 355 677H373L379 671V211H471V165H379V114Q379 73 379 66T385 54Q393 47 442 46H471V0H462ZM293 211V545L74 212L183 211H293Z"></path><path stroke-width="0" id="E2681-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2681-MJMAIN-6C" d="M42 46H56Q95 46 103 60V68Q103 77 103 91T103 124T104 167T104 217T104 272T104 329Q104 366 104 407T104 482T104 542T103 586T103 603Q100 622 89 628T44 637H26V660Q26 683 28 683L38 684Q48 685 67 686T104 688Q121 689 141 690T171 693T182 694H185V379Q185 62 186 60Q190 52 198 49Q219 46 247 46H263V0H255L232 1Q209 2 183 2T145 3T107 3T57 1L34 0H26V46H42Z"></path><path stroke-width="0" id="E2681-MJMAIN-6E" d="M41 46H55Q94 46 102 60V68Q102 77 102 91T102 122T103 161T103 203Q103 234 103 269T102 328V351Q99 370 88 376T43 385H25V408Q25 431 27 431L37 432Q47 433 65 434T102 436Q119 437 138 438T167 441T178 442H181V402Q181 364 182 364T187 369T199 384T218 402T247 421T285 437Q305 442 336 442Q450 438 463 329Q464 322 464 190V104Q464 66 466 59T477 49Q498 46 526 46H542V0H534L510 1Q487 2 460 2T422 3Q319 3 310 0H302V46H318Q379 46 379 62Q380 64 380 200Q379 335 378 343Q372 371 358 385T334 402T308 404Q263 404 229 370Q202 343 195 315T187 232V168V108Q187 78 188 68T191 55T200 49Q221 46 249 46H265V0H257L234 1Q210 2 183 2T145 3Q42 3 33 0H25V46H41Z"></path><path stroke-width="0" id="E2681-MJMAIN-2207" d="M46 676Q46 679 51 683H781Q786 679 786 676Q786 674 617 326T444 -26Q439 -33 416 -33T388 -26Q385 -22 216 326T46 676ZM697 596Q697 597 445 597T193 596Q195 591 319 336T445 80L697 596Z"></path><path stroke-width="0" id="E2681-MJMAIN-35" d="M164 157Q164 133 148 117T109 101H102Q148 22 224 22Q294 22 326 82Q345 115 345 210Q345 313 318 349Q292 382 260 382H254Q176 382 136 314Q132 307 129 306T114 304Q97 304 95 310Q93 314 93 485V614Q93 664 98 664Q100 666 102 666Q103 666 123 658T178 642T253 634Q324 634 389 662Q397 666 402 666Q410 666 410 648V635Q328 538 205 538Q174 538 149 544L139 546V374Q158 388 169 396T205 412T256 420Q337 420 393 355T449 201Q449 109 385 44T229 -22Q148 -22 99 32T50 154Q50 178 61 192T84 210T107 214Q132 214 148 197T164 157Z"></path><path stroke-width="0" id="E2681-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2681-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path><path stroke-width="0" id="E2681-MJMAIN-36" d="M42 313Q42 476 123 571T303 666Q372 666 402 630T432 550Q432 525 418 510T379 495Q356 495 341 509T326 548Q326 592 373 601Q351 623 311 626Q240 626 194 566Q147 500 147 364L148 360Q153 366 156 373Q197 433 263 433H267Q313 433 348 414Q372 400 396 374T435 317Q456 268 456 210V192Q456 169 451 149Q440 90 387 34T253 -22Q225 -22 199 -14T143 16T92 75T56 172T42 313ZM257 397Q227 397 205 380T171 335T154 278T148 216Q148 133 160 97T198 39Q222 21 251 21Q302 21 329 59Q342 77 347 104T352 209Q352 289 347 316T329 361Q302 397 257 397Z"></path><path stroke-width="0" id="E2681-MJMAIN-37" d="M55 458Q56 460 72 567L88 674Q88 676 108 676H128V672Q128 662 143 655T195 646T364 644H485V605L417 512Q408 500 387 472T360 435T339 403T319 367T305 330T292 284T284 230T278 162T275 80Q275 66 275 52T274 28V19Q270 2 255 -10T221 -22Q210 -22 200 -19T179 0T168 40Q168 198 265 368Q285 400 349 489L395 552H302Q128 552 119 546Q113 543 108 522T98 479L95 458V455H55V458Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><g transform="translate(11009,-2531)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">算</text><g transform="translate(1052,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">法</text></g><use transform="scale(1.2)" xlink:href="#E2681-MJMAINB-38" x="1963" y="0"></use><use transform="scale(1.2)" xlink:href="#E2681-MJMAINB-2D" x="2538" y="0"></use><use transform="scale(1.2)" xlink:href="#E2681-MJMAINB-31" x="2921" y="0"></use><g transform="translate(4945,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">动</text></g><g transform="translate(5961,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">作</text></g><g transform="translate(7014,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">价</text></g><g transform="translate(8067,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">值</text></g><g transform="translate(9120,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">同</text></g><g transform="translate(10136,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(11189,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(12242,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(13294,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">者</text></g><use transform="scale(1.2)" xlink:href="#E2681-MJMAINB-2F" x="11956" y="0"></use><g transform="translate(15037,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">评</text></g><g transform="translate(16090,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">论</text></g><g transform="translate(17143,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">者</text></g><g transform="translate(18196,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(19249,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">法</text></g></g><g transform="translate(0,-16352)"><g transform="translate(-19,0)"><g transform="translate(0,12290)"><g><rect fill="black" stroke="none" width="1569" height="100" x="0" y="500"></rect></g></g><g transform="translate(0,-12091)"><g><rect fill="black" stroke="none" width="1569" height="100" x="0" y="-500"></rect></g></g></g><g transform="translate(1551,0)"><g transform="translate(0,12290)"><g><rect fill="black" stroke="none" width="41597" height="100" x="0" y="500"></rect></g></g><g transform="translate(0,10990)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">输</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">入</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">：</text></g><g transform="translate(2491,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">环</text></g><g transform="translate(3322,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">境</text></g><g transform="translate(4153,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(4983,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">无</text></g><g transform="translate(5814,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">数</text></g><g transform="translate(6645,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">学</text></g><g transform="translate(7475,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">描</text></g><g transform="translate(8306,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">述</text></g><g transform="translate(9137,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(9967,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g><g transform="translate(0,9690)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">输</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">出</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">：</text></g><g transform="translate(2491,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">最</text></g><g transform="translate(3322,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">优</text></g><g transform="translate(4153,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(4983,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">略</text></g><g transform="translate(5814,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">的</text></g><g transform="translate(6645,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">估</text></g><g transform="translate(7475,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(8556,0)"><use xlink:href="#E2681-MJMATHI-3C0" x="0" y="0"></use><use xlink:href="#E2681-MJMAIN-28" x="573" y="0"></use><use xlink:href="#E2681-MJMATHI-3B8" x="962" y="0"></use><use xlink:href="#E2681-MJMAIN-29" x="1431" y="0"></use></g><g transform="translate(10376,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g><g transform="translate(0,8197)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">参</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">数</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">：</text></g><g transform="translate(2491,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">优</text></g><g transform="translate(3322,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">化</text></g><g transform="translate(4153,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">器</text></g><g transform="translate(4983,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(5814,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">隐</text></g><g transform="translate(6645,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">含</text></g><g transform="translate(7475,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">学</text></g><g transform="translate(8306,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">习</text></g><g transform="translate(9137,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">率</text></g><g transform="translate(10217,0)"><use xlink:href="#E2681-MJMATHI-3B1" x="0" y="0"></use><g transform="translate(640,412)"><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-29" x="1220" y="0"></use></g><use xlink:href="#E2681-MJMAIN-2C" x="1877" y="0"></use><g transform="translate(2322,0)"><use xlink:href="#E2681-MJMATHI-3B1" x="0" y="0"></use><g transform="translate(640,412)"><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-29" x="858" y="0"></use></g></g></g><g transform="translate(14161,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">折</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">扣</text></g><g transform="translate(3572,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">因</text></g><g transform="translate(4403,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">子</text></g></g><use xlink:href="#E2681-MJMATHI-3B3" x="19645" y="0"></use><g transform="translate(20188,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">控</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">制</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">回</text></g><g transform="translate(3572,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">合</text></g><g transform="translate(4403,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">数</text></g><g transform="translate(5233,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">和</text></g><g transform="translate(6064,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">回</text></g><g transform="translate(6895,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">合</text></g><g transform="translate(7725,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">内</text></g><g transform="translate(8556,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">步</text></g><g transform="translate(9387,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">数</text></g><g transform="translate(10217,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">的</text></g><g transform="translate(11048,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">参</text></g><g transform="translate(11879,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">数</text></g><g transform="translate(12709,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g><g transform="translate(0,6881)"><use xlink:href="#E2681-MJMAIN-31"></use><use xlink:href="#E2681-MJMAIN-2E" x="500" y="0"></use><g transform="translate(778,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(1608,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">初</text></g><g transform="translate(2439,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">始</text></g><g transform="translate(3269,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">化</text></g><g transform="translate(4100,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(4931,0)"><use xlink:href="#E2681-MJMATHI-3B8" x="0" y="0"></use><use xlink:href="#E2681-MJMAIN-2190" x="746" y="0"></use></g><g transform="translate(6678,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">任</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">意</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">值</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g></g><g transform="translate(10250,0)"><use xlink:href="#E2681-MJMAINB-77" x="0" y="0"></use><use xlink:href="#E2681-MJMAIN-2190" x="1108" y="0"></use></g><g transform="translate(12359,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">任</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">意</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">值</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g><g transform="translate(0,5581)"><use xlink:href="#E2681-MJMAIN-32"></use><use xlink:href="#E2681-MJMAIN-2E" x="500" y="0"></use><g transform="translate(778,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(1608,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">带</text></g><g transform="translate(2439,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">自</text></g><g transform="translate(3269,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">益</text></g><g transform="translate(4100,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">的</text></g><g transform="translate(4931,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(5761,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">略</text></g><g transform="translate(6592,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(7423,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(8253,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(9084,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">对</text></g><g transform="translate(9915,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">于</text></g><g transform="translate(10745,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">每</text></g><g transform="translate(11576,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">个</text></g><g transform="translate(12407,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">回</text></g><g transform="translate(13237,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">合</text></g><g transform="translate(14068,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(14899,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(15729,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">以</text></g><g transform="translate(16560,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">下</text></g><g transform="translate(17391,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">操</text></g><g transform="translate(18221,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g><g transform="translate(19052,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">：</text></g></g><g transform="translate(0,4281)"><g transform="translate(2000,0)"><use xlink:href="#E2681-MJMAIN-32"></use><use xlink:href="#E2681-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2681-MJMAIN-31" x="778" y="0"></use><g transform="translate(1278,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2108,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">初</text></g><g transform="translate(2939,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">始</text></g><g transform="translate(3769,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">化</text></g><g transform="translate(4600,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">累</text></g><g transform="translate(5431,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(6261,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">折</text></g><g transform="translate(7092,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">扣</text></g><g transform="translate(7923,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(8753,0)"><use xlink:href="#E2681-MJMATHI-49" x="0" y="0"></use><use xlink:href="#E2681-MJMAIN-2190" x="781" y="0"></use><use xlink:href="#E2681-MJMAIN-31" x="2059" y="0"></use></g><g transform="translate(11313,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g></g><g transform="translate(0,2981)"><g transform="translate(2000,0)"><use xlink:href="#E2681-MJMAIN-32"></use><use xlink:href="#E2681-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2681-MJMAIN-32" x="778" y="0"></use><g transform="translate(1278,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2108,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">决</text></g><g transform="translate(2939,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">定</text></g><g transform="translate(3769,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">初</text></g><g transform="translate(4600,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">始</text></g><g transform="translate(5431,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(6261,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g><g transform="translate(7092,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">动</text></g><g transform="translate(7923,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g><g transform="translate(8753,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">对</text></g><g transform="translate(9584,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(10415,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">选</text></g><g transform="translate(11245,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">择</text></g><g transform="translate(12076,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(12907,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g><use xlink:href="#E2681-MJMATHI-53" x="13987" y="0"></use><g transform="translate(14632,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">并</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">用</text></g></g><g transform="translate(17624,0)"><use xlink:href="#E2681-MJMATHI-3C0" x="0" y="0"></use><use xlink:href="#E2681-MJMAIN-28" x="573" y="0"></use><use xlink:href="#E2681-MJMAIN-22C5" x="962" y="0"></use><use xlink:href="#E2681-MJMAIN-2223" x="1517" y="0"></use><use xlink:href="#E2681-MJMATHI-53" x="2073" y="0"></use><use xlink:href="#E2681-MJMAIN-3B" x="2718" y="0"></use><use xlink:href="#E2681-MJMATHI-3B8" x="3163" y="0"></use><use xlink:href="#E2681-MJMAIN-29" x="3632" y="0"></use></g><g transform="translate(21645,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">得</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">到</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">动</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g></g><use xlink:href="#E2681-MJMATHI-41" x="25468" y="0"></use><g transform="translate(26218,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g></g><g transform="translate(0,1631)"><g transform="translate(2000,0)"><use xlink:href="#E2681-MJMAIN-32"></use><use xlink:href="#E2681-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2681-MJMAIN-33" x="778" y="0"></use><g transform="translate(1972,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">如</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">果</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">回</text></g><g transform="translate(2491,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">合</text></g><g transform="translate(3322,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">未</text></g><g transform="translate(4153,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">结</text></g><g transform="translate(4983,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">束</text></g><g transform="translate(5814,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(6645,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(7475,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(8306,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">以</text></g><g transform="translate(9137,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">下</text></g><g transform="translate(9967,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">操</text></g><g transform="translate(10798,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g><g transform="translate(11629,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">：</text></g></g></g></g><g transform="translate(0,322)"><g transform="translate(4000,0)"><use xlink:href="#E2681-MJMAIN-32"></use><use xlink:href="#E2681-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2681-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2681-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2681-MJMAIN-31" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">采</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">样</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">根</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">据</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g><use xlink:href="#E2681-MJMATHI-53" x="8951" y="0"></use><g transform="translate(9596,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">和</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">动</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g></g><use xlink:href="#E2681-MJMATHI-41" x="12588" y="0"></use><g transform="translate(13338,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">得</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">到</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">奖</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">励</text></g></g><use xlink:href="#E2681-MJMATHI-52" x="17160" y="0"></use><g transform="translate(17919,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">和</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">下</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">一</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">个</text></g><g transform="translate(3572,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(4403,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g></g><g transform="translate(23403,0)"><use xlink:href="#E2681-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-2032" x="925" y="583"></use></g><g transform="translate(24352,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-987)"><g transform="translate(4000,0)"><use xlink:href="#E2681-MJMAIN-32"></use><use xlink:href="#E2681-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2681-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2681-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2681-MJMAIN-32" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">用</text></g><g transform="translate(6459,0)"><use xlink:href="#E2681-MJMATHI-3C0" x="0" y="0"></use><use xlink:href="#E2681-MJMAIN-28" x="573" y="0"></use><use xlink:href="#E2681-MJMAIN-22C5" x="962" y="0"></use><use xlink:href="#E2681-MJMAIN-2223" x="1517" y="0"></use><g transform="translate(2073,0)"><use xlink:href="#E2681-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-2032" x="925" y="583"></use></g><use xlink:href="#E2681-MJMAIN-3B" x="3022" y="0"></use><use xlink:href="#E2681-MJMATHI-3B8" x="3467" y="0"></use><use xlink:href="#E2681-MJMAIN-29" x="3936" y="0"></use></g><g transform="translate(10784,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">得</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">到</text></g></g><g transform="translate(12945,0)"><use xlink:href="#E2681-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-2032" x="1060" y="583"></use></g><g transform="translate(13990,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-2346)"><g transform="translate(4000,0)"><use xlink:href="#E2681-MJMAIN-32"></use><use xlink:href="#E2681-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2681-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2681-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2681-MJMAIN-33" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">估</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">回</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">报</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><use xlink:href="#E2681-MJMATHI-55" x="0" y="0"></use><use xlink:href="#E2681-MJMAIN-2190" x="1044" y="0"></use><use xlink:href="#E2681-MJMATHI-52" x="2322" y="0"></use><use xlink:href="#E2681-MJMAIN-2B" x="3303" y="0"></use><use xlink:href="#E2681-MJMATHI-3B3" x="4304" y="0"></use><use xlink:href="#E2681-MJMATHI-71" x="4847" y="0"></use><use xlink:href="#E2681-MJMAIN-28" x="5307" y="0"></use><g transform="translate(5696,0)"><use xlink:href="#E2681-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-2032" x="925" y="583"></use></g><use xlink:href="#E2681-MJMAIN-2C" x="6645" y="0"></use><g transform="translate(7089,0)"><use xlink:href="#E2681-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-2032" x="1060" y="583"></use></g><use xlink:href="#E2681-MJMAIN-3B" x="8134" y="0"></use><use xlink:href="#E2681-MJMAINB-77" x="8578" y="0"></use><use xlink:href="#E2681-MJMAIN-29" x="9409" y="0"></use></g><g transform="translate(16838,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-3696)"><g transform="translate(4000,0)"><use xlink:href="#E2681-MJMAIN-32"></use><use xlink:href="#E2681-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2681-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2681-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2681-MJMAIN-34" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">略</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">改</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">进</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><use xlink:href="#E2681-MJMATHI-3B8" x="8951" y="0"></use><g transform="translate(9420,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">以</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">减</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">小</text></g></g><g transform="translate(12412,0)"><use xlink:href="#E2681-MJMAIN-2212" x="0" y="0"></use><use xlink:href="#E2681-MJMATHI-49" x="778" y="0"></use><use xlink:href="#E2681-MJMATHI-71" x="1282" y="0"></use><use xlink:href="#E2681-MJMAIN-28" x="1742" y="0"></use><use xlink:href="#E2681-MJMATHI-53" x="2131" y="0"></use><use xlink:href="#E2681-MJMAIN-2C" x="2776" y="0"></use><use xlink:href="#E2681-MJMATHI-41" x="3220" y="0"></use><use xlink:href="#E2681-MJMAIN-3B" x="3970" y="0"></use><use xlink:href="#E2681-MJMAINB-77" x="4415" y="0"></use><use xlink:href="#E2681-MJMAIN-29" x="5246" y="0"></use><g transform="translate(5802,0)"><use xlink:href="#E2681-MJMAIN-6C"></use><use xlink:href="#E2681-MJMAIN-6E" x="278" y="0"></use></g><use xlink:href="#E2681-MJMATHI-3C0" x="6802" y="0"></use><use xlink:href="#E2681-MJMAIN-28" x="7375" y="0"></use><use xlink:href="#E2681-MJMATHI-41" x="7764" y="0"></use><use xlink:href="#E2681-MJMAIN-2223" x="8792" y="0"></use><use xlink:href="#E2681-MJMATHI-53" x="9348" y="0"></use><use xlink:href="#E2681-MJMAIN-3B" x="9993" y="0"></use><use xlink:href="#E2681-MJMATHI-3B8" x="10437" y="0"></use><use xlink:href="#E2681-MJMAIN-29" x="10906" y="0"></use></g><g transform="translate(23708,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g></g></g></g><g transform="translate(0,-5189)"><g transform="translate(6444,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">如</text><g transform="translate(1080,0)"><use xlink:href="#E2681-MJMATHI-3B8" x="0" y="0"></use><use xlink:href="#E2681-MJMAIN-2190" x="746" y="0"></use><use xlink:href="#E2681-MJMATHI-3B8" x="2024" y="0"></use><use xlink:href="#E2681-MJMAIN-2B" x="2715" y="0"></use><g transform="translate(3716,0)"><use xlink:href="#E2681-MJMATHI-3B1" x="0" y="0"></use><g transform="translate(640,412)"><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-29" x="858" y="0"></use></g></g><use xlink:href="#E2681-MJMATHI-49" x="5337" y="0"></use><use xlink:href="#E2681-MJMATHI-71" x="5841" y="0"></use><use xlink:href="#E2681-MJMAIN-28" x="6301" y="0"></use><use xlink:href="#E2681-MJMATHI-53" x="6690" y="0"></use><use xlink:href="#E2681-MJMAIN-2C" x="7335" y="0"></use><use xlink:href="#E2681-MJMATHI-41" x="7780" y="0"></use><use xlink:href="#E2681-MJMAIN-3B" x="8530" y="0"></use><use xlink:href="#E2681-MJMAINB-77" x="8975" y="0"></use><use xlink:href="#E2681-MJMAIN-29" x="9806" y="0"></use><use xlink:href="#E2681-MJMAIN-2207" x="10195" y="0"></use><g transform="translate(11194,0)"><use xlink:href="#E2681-MJMAIN-6C"></use><use xlink:href="#E2681-MJMAIN-6E" x="278" y="0"></use></g><use xlink:href="#E2681-MJMATHI-3C0" x="12195" y="0"></use><use xlink:href="#E2681-MJMAIN-28" x="12768" y="0"></use><use xlink:href="#E2681-MJMATHI-41" x="13157" y="0"></use><use xlink:href="#E2681-MJMAIN-2223" x="14185" y="0"></use><use xlink:href="#E2681-MJMATHI-53" x="14740" y="0"></use><use xlink:href="#E2681-MJMAIN-3B" x="15385" y="0"></use><use xlink:href="#E2681-MJMATHI-3B8" x="15830" y="0"></use><use xlink:href="#E2681-MJMAIN-29" x="16299" y="0"></use></g><g transform="translate(17769,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-6623)"><g transform="translate(4000,0)"><use xlink:href="#E2681-MJMAIN-32"></use><use xlink:href="#E2681-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2681-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2681-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2681-MJMAIN-35" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">价</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">值</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><use xlink:href="#E2681-MJMAINB-77" x="8951" y="0"></use><g transform="translate(9782,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">以</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">减</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">小</text></g></g><g transform="translate(12774,0)"><use xlink:href="#E2681-MJMAIN-5B" x="0" y="0"></use><use xlink:href="#E2681-MJMATHI-55" x="278" y="0"></use><use xlink:href="#E2681-MJMAIN-2212" x="1267" y="0"></use><use xlink:href="#E2681-MJMATHI-71" x="2267" y="0"></use><use xlink:href="#E2681-MJMAIN-28" x="2727" y="0"></use><use xlink:href="#E2681-MJMATHI-53" x="3116" y="0"></use><use xlink:href="#E2681-MJMAIN-2C" x="3761" y="0"></use><use xlink:href="#E2681-MJMATHI-41" x="4206" y="0"></use><use xlink:href="#E2681-MJMAIN-3B" x="4956" y="0"></use><use xlink:href="#E2681-MJMAINB-77" x="5400" y="0"></use><use xlink:href="#E2681-MJMAIN-29" x="6231" y="0"></use><g transform="translate(6620,0)"><use xlink:href="#E2681-MJMAIN-5D" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-32" x="393" y="583"></use></g></g><g transform="translate(20126,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g></g></g></g><g transform="translate(0,-8116)"><g transform="translate(6444,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">如</text><g transform="translate(1080,0)"><use xlink:href="#E2681-MJMAINB-77" x="0" y="0"></use><use xlink:href="#E2681-MJMAIN-2190" x="1108" y="0"></use><use xlink:href="#E2681-MJMAINB-77" x="2386" y="0"></use><use xlink:href="#E2681-MJMAIN-2B" x="3439" y="0"></use><g transform="translate(4440,0)"><use xlink:href="#E2681-MJMATHI-3B1" x="0" y="0"></use><g transform="translate(640,412)"><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-29" x="1220" y="0"></use></g></g><use xlink:href="#E2681-MJMAIN-5B" x="6317" y="0"></use><use xlink:href="#E2681-MJMATHI-55" x="6595" y="0"></use><use xlink:href="#E2681-MJMAIN-2212" x="7584" y="0"></use><use xlink:href="#E2681-MJMATHI-71" x="8585" y="0"></use><use xlink:href="#E2681-MJMAIN-28" x="9045" y="0"></use><use xlink:href="#E2681-MJMATHI-53" x="9434" y="0"></use><use xlink:href="#E2681-MJMAIN-2C" x="10079" y="0"></use><use xlink:href="#E2681-MJMATHI-41" x="10523" y="0"></use><use xlink:href="#E2681-MJMAIN-3B" x="11273" y="0"></use><use xlink:href="#E2681-MJMAINB-77" x="11718" y="0"></use><use xlink:href="#E2681-MJMAIN-29" x="12549" y="0"></use><use xlink:href="#E2681-MJMAIN-5D" x="12938" y="0"></use><use xlink:href="#E2681-MJMAIN-2207" x="13216" y="0"></use><use xlink:href="#E2681-MJMATHI-71" x="14049" y="0"></use><use xlink:href="#E2681-MJMAIN-28" x="14509" y="0"></use><use xlink:href="#E2681-MJMATHI-53" x="14898" y="0"></use><use xlink:href="#E2681-MJMAIN-2C" x="15543" y="0"></use><use xlink:href="#E2681-MJMATHI-41" x="15988" y="0"></use><use xlink:href="#E2681-MJMAIN-3B" x="16738" y="0"></use><use xlink:href="#E2681-MJMAINB-77" x="17182" y="0"></use><use xlink:href="#E2681-MJMAIN-29" x="18013" y="0"></use></g><g transform="translate(19483,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-9466)"><g transform="translate(4000,0)"><use xlink:href="#E2681-MJMAIN-32"></use><use xlink:href="#E2681-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2681-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2681-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2681-MJMAIN-36" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">累</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">折</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">扣</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(8701,0)"><use xlink:href="#E2681-MJMATHI-49" x="0" y="0"></use><use xlink:href="#E2681-MJMAIN-2190" x="781" y="0"></use><use xlink:href="#E2681-MJMATHI-3B3" x="2059" y="0"></use><use xlink:href="#E2681-MJMATHI-49" x="2602" y="0"></use></g><g transform="translate(11807,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-10791)"><g transform="translate(4000,0)"><use xlink:href="#E2681-MJMAIN-32"></use><use xlink:href="#E2681-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2681-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2681-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2681-MJMAIN-37" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><use xlink:href="#E2681-MJMATHI-53" x="0" y="0"></use><use xlink:href="#E2681-MJMAIN-2190" x="922" y="0"></use><g transform="translate(2200,0)"><use xlink:href="#E2681-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-2032" x="925" y="583"></use></g><use xlink:href="#E2681-MJMAIN-2C" x="3149" y="0"></use><use xlink:href="#E2681-MJMATHI-41" x="3594" y="0"></use><use xlink:href="#E2681-MJMAIN-2190" x="4622" y="0"></use><g transform="translate(5899,0)"><use xlink:href="#E2681-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2681-MJMAIN-2032" x="1060" y="583"></use></g></g><g transform="translate(13984,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g></g><g transform="translate(0,-12091)"><g><rect fill="black" stroke="none" width="41597" height="100" x="0" y="-500"></rect></g></g></g></g></g></svg></span></div><script type="math/tex; mode=display" id="MathJax-Element-1984">\; \\ \; \\
\large \textbf{算法 8-1   动作价值同策执行者/评论者算法} \\
\begin{split}
\rule[5pt]{10mm}{0.1em} &\rule[5pt]{265mm}{0.1em} \\
&\text{输入：环境（无数学描述）。} \\
&\text{输出：最优策略的估计 $\pi(\theta)$ 。} \\
&\text{参数：优化器（隐含学习率 $\alpha^{(\bold w)}, \alpha^{(\theta)}$ ），折扣因子 $\gamma$ ，控制回合数和回合内步数的参数。} \\
&\text{1.（初始化）$\theta \leftarrow$ 任意值，$\bold w \leftarrow$ 任意值。} \\
&\text{2.（带自益的策略更新）对于每个回合执行以下操作：} \\
&\qquad \text{2.1（初始化累计折扣）$I \leftarrow 1$ 。} \\
&\qquad \text{2.2（决定初始状态动作对）选择状态 $S$ ，并用 $\pi(\cdot \mid S; \theta)$ 得到动作 $A$ 。} \\
&\qquad \text{2.3 $\;\,$如果回合未结束，执行以下操作：} \\
&\qquad \qquad \text{2.3.1（采样）根据状态 $S$ 和动作 $A$ 得到奖励 $R$ 和下一个状态 $S'$ ；} \\
&\qquad \qquad \text{2.3.2（执行）用 $\pi(\cdot \mid S'; \theta)$ 得到 $A'$ ；} \\
&\qquad \qquad \text{2.3.3（估计回报）$U \leftarrow R + \gamma q(S',A';\bold w)$ ；} \\
&\qquad \qquad \text{2.3.4（策略改进）更新 $\theta$ 以减小 $-Iq(S,A;\bold w) \ln \pi(A \mid S; \theta)$ ，} \\
&\qquad \qquad \qquad \;\, \text{如 $\theta \leftarrow \theta + \alpha^{(\theta)} Iq(S,A;\bold w) \nabla \ln \pi(A \mid S; \theta)$ ；} \\
&\qquad \qquad \text{2.3.5（更新价值）更新 $\bold w$ 以减小 $[U - q(S,A;\bold w)]^2$ ，} \\
&\qquad \qquad \qquad \;\, \text{如 $\bold w \leftarrow \bold w + \alpha^{(\bold w)} [U - q(S,A;\bold w)] \nabla q(S,A;\bold w)$ ；} \\
&\qquad \qquad \text{2.3.6（更新累计折扣）$I \leftarrow \gamma I$ ；} \\
&\qquad \qquad \text{2.3.7（更新状态）$S \leftarrow S', A \leftarrow A'$ 。} \\
\rule[-5pt]{10mm}{0.1em} &\rule[-5pt]{265mm}{0.1em}
\end{split}
\; \\ \; \\</script>
					</div></div><p><span>当使用 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="11.687ex" height="2.71ex" viewBox="0 -832.7 5031.9 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2701-MJMATHI-71" d="M33 157Q33 258 109 349T280 441Q340 441 372 389Q373 390 377 395T388 406T404 418Q438 442 450 442Q454 442 457 439T460 434Q460 425 391 149Q320 -135 320 -139Q320 -147 365 -148H390Q396 -156 396 -157T393 -175Q389 -188 383 -194H370Q339 -192 262 -192Q234 -192 211 -192T174 -192T157 -193Q143 -193 143 -185Q143 -182 145 -170Q149 -154 152 -151T172 -148Q220 -148 230 -141Q238 -136 258 -53T279 32Q279 33 272 29Q224 -10 172 -10Q117 -10 75 30T33 157ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"></path><path stroke-width="0" id="E2701-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2701-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2701-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2701-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2701-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2701-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2701-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2701-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2701-MJMATHI-71" x="0" y="0"></use><use xlink:href="#E2701-MJMAIN-28" x="460" y="0"></use><g transform="translate(849,0)"><use xlink:href="#E2701-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2701-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2701-MJMAIN-2C" x="1817" y="0"></use><g transform="translate(2261,0)"><use xlink:href="#E2701-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2701-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2701-MJMAIN-3B" x="3367" y="0"></use><use xlink:href="#E2701-MJMAINB-77" x="3811" y="0"></use><use xlink:href="#E2701-MJMAIN-29" x="4642" y="0"></use></g></svg></span><script type="math/tex">q(S_t,A_t;\bold w)</script><span> 作为回报估计，并取 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="31.731ex" height="3ex" viewBox="0 -874.2 13661.9 1291.5" role="img" focusable="false" style="vertical-align: -0.969ex;"><defs><path stroke-width="0" id="E2702-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2702-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2702-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path stroke-width="0" id="E2702-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2702-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2702-MJMATHI-71" d="M33 157Q33 258 109 349T280 441Q340 441 372 389Q373 390 377 395T388 406T404 418Q438 442 450 442Q454 442 457 439T460 434Q460 425 391 149Q320 -135 320 -139Q320 -147 365 -148H390Q396 -156 396 -157T393 -175Q389 -188 383 -194H370Q339 -192 262 -192Q234 -192 211 -192T174 -192T157 -193Q143 -193 143 -185Q143 -182 145 -170Q149 -154 152 -151T172 -148Q220 -148 230 -141Q238 -136 258 -53T279 32Q279 33 272 29Q224 -10 172 -10Q117 -10 75 30T33 157ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"></path><path stroke-width="0" id="E2702-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2702-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2702-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2702-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2702-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2702-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2702-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2702-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2702-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2702-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2702-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2702-MJMATHI-74" x="1100" y="-213"></use><use xlink:href="#E2702-MJMAIN-3D" x="1411" y="0"></use><g transform="translate(2466,0)"><use xlink:href="#E2702-MJMATHI-3B3" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2702-MJMATHI-74" x="778" y="513"></use></g><use xlink:href="#E2702-MJMAIN-5B" x="3372" y="0"></use><g transform="translate(3650,0)"><use xlink:href="#E2702-MJMATHI-71" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2702-MJMAIN-28" x="630" y="-263"></use></g><g transform="translate(4471,0)"><use xlink:href="#E2702-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2702-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2702-MJMAIN-2C" x="5439" y="0"></use><g transform="translate(5884,0)"><use xlink:href="#E2702-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2702-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2702-MJMAIN-3B" x="6989" y="0"></use><use xlink:href="#E2702-MJMAINB-77" x="7434" y="0"></use><use xlink:href="#E2702-MJMAIN-29" x="8265" y="0"></use><use xlink:href="#E2702-MJMAIN-2212" x="8876" y="0"></use><use xlink:href="#E2702-MJMATHI-76" x="9877" y="0"></use><use xlink:href="#E2702-MJMAIN-28" x="10362" y="0"></use><g transform="translate(10751,0)"><use xlink:href="#E2702-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2702-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2702-MJMAIN-3B" x="11719" y="0"></use><use xlink:href="#E2702-MJMAINB-77" x="12163" y="0"></use><use xlink:href="#E2702-MJMAIN-29" x="12994" y="0"></use><use xlink:href="#E2702-MJMAIN-5D" x="13383" y="0"></use></g></svg></span><script type="math/tex">\Psi_t = \gamma^t[q_(S_t,A_t;\bold w)-v(S_t;\bold w)]</script><span> 时的同策算法称为优势执行者 / 评论者算法；相当于在基本的执行者 / 评论者算法中引入基线函数 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="17.061ex" height="2.71ex" viewBox="0 -832.7 7345.8 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2703-MJMATHI-42" d="M231 637Q204 637 199 638T194 649Q194 676 205 682Q206 683 335 683Q594 683 608 681Q671 671 713 636T756 544Q756 480 698 429T565 360L555 357Q619 348 660 311T702 219Q702 146 630 78T453 1Q446 0 242 0Q42 0 39 2Q35 5 35 10Q35 17 37 24Q42 43 47 45Q51 46 62 46H68Q95 46 128 49Q142 52 147 61Q150 65 219 339T288 628Q288 635 231 637ZM649 544Q649 574 634 600T585 634Q578 636 493 637Q473 637 451 637T416 636H403Q388 635 384 626Q382 622 352 506Q352 503 351 500L320 374H401Q482 374 494 376Q554 386 601 434T649 544ZM595 229Q595 273 572 302T512 336Q506 337 429 337Q311 337 310 336Q310 334 293 263T258 122L240 52Q240 48 252 48T333 46Q422 46 429 47Q491 54 543 105T595 229Z"></path><path stroke-width="0" id="E2703-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2703-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2703-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2703-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2703-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path stroke-width="0" id="E2703-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2703-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2703-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2703-MJMATHI-42" x="0" y="0"></use><use xlink:href="#E2703-MJMAIN-28" x="759" y="0"></use><g transform="translate(1148,0)"><use xlink:href="#E2703-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2703-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2703-MJMAIN-29" x="2116" y="0"></use><use xlink:href="#E2703-MJMAIN-3D" x="2783" y="0"></use><use xlink:href="#E2703-MJMATHI-76" x="3838" y="0"></use><use xlink:href="#E2703-MJMAIN-28" x="4323" y="0"></use><g transform="translate(4712,0)"><use xlink:href="#E2703-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2703-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2703-MJMAIN-3B" x="5681" y="0"></use><use xlink:href="#E2703-MJMAINB-77" x="6125" y="0"></use><use xlink:href="#E2703-MJMAIN-29" x="6956" y="0"></use></g></svg></span><script type="math/tex">B(S_t)=v(S_t;\bold w)</script><span> 得到的，其中 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="22.671ex" height="2.71ex" viewBox="0 -832.7 9761.2 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2705-MJMATHI-71" d="M33 157Q33 258 109 349T280 441Q340 441 372 389Q373 390 377 395T388 406T404 418Q438 442 450 442Q454 442 457 439T460 434Q460 425 391 149Q320 -135 320 -139Q320 -147 365 -148H390Q396 -156 396 -157T393 -175Q389 -188 383 -194H370Q339 -192 262 -192Q234 -192 211 -192T174 -192T157 -193Q143 -193 143 -185Q143 -182 145 -170Q149 -154 152 -151T172 -148Q220 -148 230 -141Q238 -136 258 -53T279 32Q279 33 272 29Q224 -10 172 -10Q117 -10 75 30T33 157ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"></path><path stroke-width="0" id="E2705-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2705-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2705-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2705-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2705-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2705-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2705-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2705-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2705-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2705-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2705-MJMATHI-71" x="0" y="0"></use><use xlink:href="#E2705-MJMAIN-28" x="460" y="0"></use><g transform="translate(849,0)"><use xlink:href="#E2705-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2705-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2705-MJMAIN-2C" x="1817" y="0"></use><g transform="translate(2261,0)"><use xlink:href="#E2705-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2705-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2705-MJMAIN-3B" x="3367" y="0"></use><use xlink:href="#E2705-MJMAINB-77" x="3811" y="0"></use><use xlink:href="#E2705-MJMAIN-29" x="4642" y="0"></use><use xlink:href="#E2705-MJMAIN-2212" x="5254" y="0"></use><use xlink:href="#E2705-MJMATHI-76" x="6254" y="0"></use><use xlink:href="#E2705-MJMAIN-28" x="6739" y="0"></use><g transform="translate(7128,0)"><use xlink:href="#E2705-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2705-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2705-MJMAIN-3B" x="8096" y="0"></use><use xlink:href="#E2705-MJMAINB-77" x="8541" y="0"></use><use xlink:href="#E2705-MJMAIN-29" x="9372" y="0"></use></g></svg></span><script type="math/tex">q(S_t,A_t;\bold w)-v(S_t;\bold w)</script><span> 是优势函数的估计。由于采用 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="22.671ex" height="2.71ex" viewBox="0 -832.7 9761.2 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2705-MJMATHI-71" d="M33 157Q33 258 109 349T280 441Q340 441 372 389Q373 390 377 395T388 406T404 418Q438 442 450 442Q454 442 457 439T460 434Q460 425 391 149Q320 -135 320 -139Q320 -147 365 -148H390Q396 -156 396 -157T393 -175Q389 -188 383 -194H370Q339 -192 262 -192Q234 -192 211 -192T174 -192T157 -193Q143 -193 143 -185Q143 -182 145 -170Q149 -154 152 -151T172 -148Q220 -148 230 -141Q238 -136 258 -53T279 32Q279 33 272 29Q224 -10 172 -10Q117 -10 75 30T33 157ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"></path><path stroke-width="0" id="E2705-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2705-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2705-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2705-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2705-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2705-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2705-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2705-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2705-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2705-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2705-MJMATHI-71" x="0" y="0"></use><use xlink:href="#E2705-MJMAIN-28" x="460" y="0"></use><g transform="translate(849,0)"><use xlink:href="#E2705-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2705-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2705-MJMAIN-2C" x="1817" y="0"></use><g transform="translate(2261,0)"><use xlink:href="#E2705-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2705-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2705-MJMAIN-3B" x="3367" y="0"></use><use xlink:href="#E2705-MJMAINB-77" x="3811" y="0"></use><use xlink:href="#E2705-MJMAIN-29" x="4642" y="0"></use><use xlink:href="#E2705-MJMAIN-2212" x="5254" y="0"></use><use xlink:href="#E2705-MJMATHI-76" x="6254" y="0"></use><use xlink:href="#E2705-MJMAIN-28" x="6739" y="0"></use><g transform="translate(7128,0)"><use xlink:href="#E2705-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2705-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2705-MJMAIN-3B" x="8096" y="0"></use><use xlink:href="#E2705-MJMAINB-77" x="8541" y="0"></use><use xlink:href="#E2705-MJMAIN-29" x="9372" y="0"></use></g></svg></span><script type="math/tex">q(S_t,A_t;\bold w)-v(S_t;\bold w)</script><span> 估计优势函数需要两个函数分别表示 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="4.805ex" height="2.71ex" viewBox="0 -832.7 2069 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2706-MJMATHI-71" d="M33 157Q33 258 109 349T280 441Q340 441 372 389Q373 390 377 395T388 406T404 418Q438 442 450 442Q454 442 457 439T460 434Q460 425 391 149Q320 -135 320 -139Q320 -147 365 -148H390Q396 -156 396 -157T393 -175Q389 -188 383 -194H370Q339 -192 262 -192Q234 -192 211 -192T174 -192T157 -193Q143 -193 143 -185Q143 -182 145 -170Q149 -154 152 -151T172 -148Q220 -148 230 -141Q238 -136 258 -53T279 32Q279 33 272 29Q224 -10 172 -10Q117 -10 75 30T33 157ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"></path><path stroke-width="0" id="E2706-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2706-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2706-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2706-MJMATHI-71" x="0" y="0"></use><use xlink:href="#E2706-MJMAIN-28" x="460" y="0"></use><use xlink:href="#E2706-MJMAINB-77" x="849" y="0"></use><use xlink:href="#E2706-MJMAIN-29" x="1680" y="0"></use></g></svg></span><script type="math/tex">q(\bold w)</script><span> 和 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="4.864ex" height="2.71ex" viewBox="0 -832.7 2094 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2707-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2707-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2707-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2707-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2707-MJMATHI-76" x="0" y="0"></use><use xlink:href="#E2707-MJMAIN-28" x="485" y="0"></use><use xlink:href="#E2707-MJMAINB-77" x="874" y="0"></use><use xlink:href="#E2707-MJMAIN-29" x="1705" y="0"></use></g></svg></span><script type="math/tex">v(\bold w)</script><span> ，为了简化，可以使用 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="24.54ex" height="2.71ex" viewBox="0 -832.7 10565.8 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2708-MJMATHI-55" d="M107 637Q73 637 71 641Q70 643 70 649Q70 673 81 682Q83 683 98 683Q139 681 234 681Q268 681 297 681T342 682T362 682Q378 682 378 672Q378 670 376 658Q371 641 366 638H364Q362 638 359 638T352 638T343 637T334 637Q295 636 284 634T266 623Q265 621 238 518T184 302T154 169Q152 155 152 140Q152 86 183 55T269 24Q336 24 403 69T501 205L552 406Q599 598 599 606Q599 633 535 637Q511 637 511 648Q511 650 513 660Q517 676 519 679T529 683Q532 683 561 682T645 680Q696 680 723 681T752 682Q767 682 767 672Q767 650 759 642Q756 637 737 637Q666 633 648 597Q646 592 598 404Q557 235 548 205Q515 105 433 42T263 -22Q171 -22 116 34T60 167V183Q60 201 115 421Q164 622 164 628Q164 635 107 637Z"></path><path stroke-width="0" id="E2708-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2708-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path stroke-width="0" id="E2708-MJMATHI-52" d="M230 637Q203 637 198 638T193 649Q193 676 204 682Q206 683 378 683Q550 682 564 680Q620 672 658 652T712 606T733 563T739 529Q739 484 710 445T643 385T576 351T538 338L545 333Q612 295 612 223Q612 212 607 162T602 80V71Q602 53 603 43T614 25T640 16Q668 16 686 38T712 85Q717 99 720 102T735 105Q755 105 755 93Q755 75 731 36Q693 -21 641 -21H632Q571 -21 531 4T487 82Q487 109 502 166T517 239Q517 290 474 313Q459 320 449 321T378 323H309L277 193Q244 61 244 59Q244 55 245 54T252 50T269 48T302 46H333Q339 38 339 37T336 19Q332 6 326 0H311Q275 2 180 2Q146 2 117 2T71 2T50 1Q33 1 33 10Q33 12 36 24Q41 43 46 45Q50 46 61 46H67Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628Q287 635 230 637ZM630 554Q630 586 609 608T523 636Q521 636 500 636T462 637H440Q393 637 386 627Q385 624 352 494T319 361Q319 360 388 360Q466 361 492 367Q556 377 592 426Q608 449 619 486T630 554Z"></path><path stroke-width="0" id="E2708-MJMAIN-2B" d="M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"></path><path stroke-width="0" id="E2708-MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="0" id="E2708-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2708-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2708-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2708-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2708-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2708-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2708-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2708-MJMATHI-55" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2708-MJMATHI-74" x="965" y="-213"></use><use xlink:href="#E2708-MJMAIN-3D" x="1316" y="0"></use><g transform="translate(2371,0)"><use xlink:href="#E2708-MJMATHI-52" x="0" y="0"></use><g transform="translate(759,-150)"><use transform="scale(0.707)" xlink:href="#E2708-MJMATHI-74" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2708-MJMAIN-2B" x="361" y="0"></use><use transform="scale(0.707)" xlink:href="#E2708-MJMAIN-31" x="1139" y="0"></use></g></g><use xlink:href="#E2708-MJMAIN-2B" x="4611" y="0"></use><use xlink:href="#E2708-MJMATHI-3B3" x="5612" y="0"></use><use xlink:href="#E2708-MJMATHI-76" x="6155" y="0"></use><use xlink:href="#E2708-MJMAIN-28" x="6640" y="0"></use><g transform="translate(7029,0)"><use xlink:href="#E2708-MJMATHI-53" x="0" y="0"></use><g transform="translate(613,-150)"><use transform="scale(0.707)" xlink:href="#E2708-MJMATHI-74" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2708-MJMAIN-2B" x="361" y="0"></use><use transform="scale(0.707)" xlink:href="#E2708-MJMAIN-31" x="1139" y="0"></use></g></g><use xlink:href="#E2708-MJMAIN-3B" x="8901" y="0"></use><use xlink:href="#E2708-MJMAINB-77" x="9345" y="0"></use><use xlink:href="#E2708-MJMAIN-29" x="10176" y="0"></use></g></svg></span><script type="math/tex">U_t = R_{t+1} + \gamma v(S_{t+1};\bold w)</script><span> 作为目标，这样优势函数的估计就变为单步时序差分的形式 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="30.016ex" height="2.71ex" viewBox="0 -832.7 12923.4 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2709-MJMATHI-52" d="M230 637Q203 637 198 638T193 649Q193 676 204 682Q206 683 378 683Q550 682 564 680Q620 672 658 652T712 606T733 563T739 529Q739 484 710 445T643 385T576 351T538 338L545 333Q612 295 612 223Q612 212 607 162T602 80V71Q602 53 603 43T614 25T640 16Q668 16 686 38T712 85Q717 99 720 102T735 105Q755 105 755 93Q755 75 731 36Q693 -21 641 -21H632Q571 -21 531 4T487 82Q487 109 502 166T517 239Q517 290 474 313Q459 320 449 321T378 323H309L277 193Q244 61 244 59Q244 55 245 54T252 50T269 48T302 46H333Q339 38 339 37T336 19Q332 6 326 0H311Q275 2 180 2Q146 2 117 2T71 2T50 1Q33 1 33 10Q33 12 36 24Q41 43 46 45Q50 46 61 46H67Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628Q287 635 230 637ZM630 554Q630 586 609 608T523 636Q521 636 500 636T462 637H440Q393 637 386 627Q385 624 352 494T319 361Q319 360 388 360Q466 361 492 367Q556 377 592 426Q608 449 619 486T630 554Z"></path><path stroke-width="0" id="E2709-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2709-MJMAIN-2B" d="M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"></path><path stroke-width="0" id="E2709-MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="0" id="E2709-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2709-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2709-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2709-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2709-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2709-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2709-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2709-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2709-MJMATHI-52" x="0" y="0"></use><g transform="translate(759,-150)"><use transform="scale(0.707)" xlink:href="#E2709-MJMATHI-74" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2709-MJMAIN-2B" x="361" y="0"></use><use transform="scale(0.707)" xlink:href="#E2709-MJMAIN-31" x="1139" y="0"></use></g><use xlink:href="#E2709-MJMAIN-2B" x="2240" y="0"></use><use xlink:href="#E2709-MJMATHI-3B3" x="3240" y="0"></use><use xlink:href="#E2709-MJMATHI-76" x="3783" y="0"></use><use xlink:href="#E2709-MJMAIN-28" x="4268" y="0"></use><g transform="translate(4657,0)"><use xlink:href="#E2709-MJMATHI-53" x="0" y="0"></use><g transform="translate(613,-150)"><use transform="scale(0.707)" xlink:href="#E2709-MJMATHI-74" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2709-MJMAIN-2B" x="361" y="0"></use><use transform="scale(0.707)" xlink:href="#E2709-MJMAIN-31" x="1139" y="0"></use></g></g><use xlink:href="#E2709-MJMAIN-3B" x="6529" y="0"></use><use xlink:href="#E2709-MJMAINB-77" x="6974" y="0"></use><use xlink:href="#E2709-MJMAIN-29" x="7805" y="0"></use><use xlink:href="#E2709-MJMAIN-2212" x="8416" y="0"></use><use xlink:href="#E2709-MJMATHI-76" x="9416" y="0"></use><use xlink:href="#E2709-MJMAIN-28" x="9901" y="0"></use><g transform="translate(10290,0)"><use xlink:href="#E2709-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2709-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2709-MJMAIN-3B" x="11258" y="0"></use><use xlink:href="#E2709-MJMAINB-77" x="11703" y="0"></use><use xlink:href="#E2709-MJMAIN-29" x="12534" y="0"></use></g></svg></span><script type="math/tex">R_{t+1} + \gamma v(S_{t+1};\bold w) - v(S_t;\bold w)</script><span> 。</span></p><p><span>如果优势执行者 / 评论者算法在执行过程中不是每一步都更新参数，而是在回合结束后用整个轨迹来进行更新，就可以把算法分为经验搜集和经验使用两个部分，这样的分隔可以让该算法同时有很多执行者在同时执行。每个执行者在一定的时机更新参数，同时更新策略参数 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="1.089ex" height="1.939ex" viewBox="0 -749.6 469 834.7" role="img" focusable="false" style="vertical-align: -0.198ex;"><defs><path stroke-width="0" id="E2716-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2716-MJMATHI-3B8" x="0" y="0"></use></g></svg></span><script type="math/tex">\theta</script><span> 和价值参数 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="1.93ex" height="1.36ex" viewBox="0 -500.4 831 585.5" role="img" focusable="false" style="vertical-align: -0.198ex;"><defs><path stroke-width="0" id="E2711-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2711-MJMAINB-77" x="0" y="0"></use></g></svg></span><script type="math/tex">\bold w</script><span> ，每个执行者的更新都是异步的，所以这样的并行算法称为</span><strong><span>异步优势执行者 / 评论者算法</span></strong><span>（Asynchronous Advantage Actor-Critic, A3C）。异步优势执行者 / 评论者算法中的自益部分，不仅可以采用单步时序差分，也可以使用多步时序差分；另外，还可以对函数参数的访问进行控制，使得所有执行者统一更新参数，这样的并行算法称为</span><strong><span>优势执行者 / 评论者算法</span></strong><span>（Advantage Actor-Critic, A2C）。</span></p><p><span>算法 8-2 和算法 8-3 分别给出了这两个算法</span><em><span>（书中给出的 8-2 优势执行者 / 评论者算法似乎不是上面所提到的并行算法）</span></em><span>：</span></p><div contenteditable="false" spellcheck="false" class="mathjax-block md-end-block md-math-block md-rawblock" id="mathjax-n18" cid="n18" mdtype="math_block"><div class="md-rawblock-container md-math-container" contenteditable="false" tabindex="-1">
						<div class="MathJax_SVG_Display"><span class="MathJax_SVG" id="MathJax-Element-1988-Frame" tabindex="-1" style="font-size: 100%; display: inline-block; zoom: 0.971345;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="101.294ex" height="52.196ex" viewBox="-18.1 -43.5 43612.5 22473.1" role="img" focusable="false" style="vertical-align: -52.095ex; margin-left: -0.042ex; max-width: 100%;"><defs><path stroke-width="0" id="E2721-MJMAINB-38" d="M80 474Q80 561 139 607T278 654Q357 654 411 632Q490 593 494 509Q494 424 416 376L407 371L418 364Q432 356 447 345T481 312T513 260T526 192Q526 100 461 45T285 -11Q184 -11 116 32T48 164Q48 181 50 196T58 225T69 249T84 270T100 286T117 300T134 311T149 321T162 329L152 336Q120 360 100 397T80 474ZM347 404Q404 446 404 503Q404 579 317 599Q309 600 276 600Q178 600 170 538Q170 532 171 527T173 518T178 509T184 501T194 492T205 484T219 476T235 467T254 456T275 445L347 404ZM289 47Q323 47 351 54T402 82T425 137Q425 147 421 161Q411 183 391 197T303 249Q224 293 223 293Q220 291 215 288T197 273T175 248T157 213T149 167Q149 109 188 78T289 47Z"></path><path stroke-width="0" id="E2721-MJMAINB-2D" d="M13 166V278H318V166H13Z"></path><path stroke-width="0" id="E2721-MJMAINB-32" d="M175 580Q175 578 185 572T205 551T215 510Q215 467 191 449T137 430Q107 430 83 448T58 511Q58 558 91 592T168 640T259 654Q328 654 383 637Q451 610 484 563T517 459Q517 401 482 360T368 262Q340 243 265 184L210 140H274Q416 140 429 145Q439 148 447 186T455 237H517V233Q516 230 501 119Q489 9 486 4V0H57V25Q57 51 58 54Q60 57 109 106T215 214T288 291Q364 377 364 458Q364 515 328 553T231 592Q214 592 201 589T181 584T175 580Z"></path><path stroke-width="0" id="E2721-MJMAINB-2F" d="M451 730Q460 750 479 750Q492 750 501 740T510 718Q508 708 318 244L122 -232Q112 -250 95 -250Q82 -250 73 -241T64 -218Q66 -205 258 261T451 730Z"></path><path stroke-width="0" id="E2721-MJMAIN-22EF" d="M78 250Q78 274 95 292T138 310Q162 310 180 294T199 251Q199 226 182 208T139 190T96 207T78 250ZM525 250Q525 274 542 292T585 310Q609 310 627 294T646 251Q646 226 629 208T586 190T543 207T525 250ZM972 250Q972 274 989 292T1032 310Q1056 310 1074 294T1093 251Q1093 226 1076 208T1033 190T990 207T972 250Z"></path><path stroke-width="0" id="E2721-MJMAIN-38" d="M70 417T70 494T124 618T248 666Q319 666 374 624T429 515Q429 485 418 459T392 417T361 389T335 371T324 363L338 354Q352 344 366 334T382 323Q457 264 457 174Q457 95 399 37T249 -22Q159 -22 101 29T43 155Q43 263 172 335L154 348Q133 361 127 368Q70 417 70 494ZM286 386L292 390Q298 394 301 396T311 403T323 413T334 425T345 438T355 454T364 471T369 491T371 513Q371 556 342 586T275 624Q268 625 242 625Q201 625 165 599T128 534Q128 511 141 492T167 463T217 431Q224 426 228 424L286 386ZM250 21Q308 21 350 55T392 137Q392 154 387 169T375 194T353 216T330 234T301 253T274 270Q260 279 244 289T218 306L210 311Q204 311 181 294T133 239T107 157Q107 98 150 60T250 21Z"></path><path stroke-width="0" id="E2721-MJMAIN-2D" d="M11 179V252H277V179H11Z"></path><path stroke-width="0" id="E2721-MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="0" id="E2721-MJMAIN-32" d="M109 429Q82 429 66 447T50 491Q50 562 103 614T235 666Q326 666 387 610T449 465Q449 422 429 383T381 315T301 241Q265 210 201 149L142 93L218 92Q375 92 385 97Q392 99 409 186V189H449V186Q448 183 436 95T421 3V0H50V19V31Q50 38 56 46T86 81Q115 113 136 137Q145 147 170 174T204 211T233 244T261 278T284 308T305 340T320 369T333 401T340 431T343 464Q343 527 309 573T212 619Q179 619 154 602T119 569T109 550Q109 549 114 549Q132 549 151 535T170 489Q170 464 154 447T109 429Z"></path><path stroke-width="0" id="E2721-MJMAIN-2E" d="M78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60Z"></path><path stroke-width="0" id="E2721-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2721-MJMAIN-33" d="M127 463Q100 463 85 480T69 524Q69 579 117 622T233 665Q268 665 277 664Q351 652 390 611T430 522Q430 470 396 421T302 350L299 348Q299 347 308 345T337 336T375 315Q457 262 457 175Q457 96 395 37T238 -22Q158 -22 100 21T42 130Q42 158 60 175T105 193Q133 193 151 175T169 130Q169 119 166 110T159 94T148 82T136 74T126 70T118 67L114 66Q165 21 238 21Q293 21 321 74Q338 107 338 175V195Q338 290 274 322Q259 328 213 329L171 330L168 332Q166 335 166 348Q166 366 174 366Q202 366 232 371Q266 376 294 413T322 525V533Q322 590 287 612Q265 626 240 626Q208 626 181 615T143 592T132 580H135Q138 579 143 578T153 573T165 566T175 555T183 540T186 520Q186 498 172 481T127 463Z"></path><path stroke-width="0" id="E2721-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2721-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2721-MJMAIN-22C5" d="M78 250Q78 274 95 292T138 310Q162 310 180 294T199 251Q199 226 182 208T139 190T96 207T78 250Z"></path><path stroke-width="0" id="E2721-MJMAIN-2223" d="M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z"></path><path stroke-width="0" id="E2721-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2721-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path><path stroke-width="0" id="E2721-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2721-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2721-MJMATHI-52" d="M230 637Q203 637 198 638T193 649Q193 676 204 682Q206 683 378 683Q550 682 564 680Q620 672 658 652T712 606T733 563T739 529Q739 484 710 445T643 385T576 351T538 338L545 333Q612 295 612 223Q612 212 607 162T602 80V71Q602 53 603 43T614 25T640 16Q668 16 686 38T712 85Q717 99 720 102T735 105Q755 105 755 93Q755 75 731 36Q693 -21 641 -21H632Q571 -21 531 4T487 82Q487 109 502 166T517 239Q517 290 474 313Q459 320 449 321T378 323H309L277 193Q244 61 244 59Q244 55 245 54T252 50T269 48T302 46H333Q339 38 339 37T336 19Q332 6 326 0H311Q275 2 180 2Q146 2 117 2T71 2T50 1Q33 1 33 10Q33 12 36 24Q41 43 46 45Q50 46 61 46H67Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628Q287 635 230 637ZM630 554Q630 586 609 608T523 636Q521 636 500 636T462 637H440Q393 637 386 627Q385 624 352 494T319 361Q319 360 388 360Q466 361 492 367Q556 377 592 426Q608 449 619 486T630 554Z"></path><path stroke-width="0" id="E2721-MJMAIN-2032" d="M79 43Q73 43 52 49T30 61Q30 68 85 293T146 528Q161 560 198 560Q218 560 240 545T262 501Q262 496 260 486Q259 479 173 263T84 45T79 43Z"></path><path stroke-width="0" id="E2721-MJMATHI-55" d="M107 637Q73 637 71 641Q70 643 70 649Q70 673 81 682Q83 683 98 683Q139 681 234 681Q268 681 297 681T342 682T362 682Q378 682 378 672Q378 670 376 658Q371 641 366 638H364Q362 638 359 638T352 638T343 637T334 637Q295 636 284 634T266 623Q265 621 238 518T184 302T154 169Q152 155 152 140Q152 86 183 55T269 24Q336 24 403 69T501 205L552 406Q599 598 599 606Q599 633 535 637Q511 637 511 648Q511 650 513 660Q517 676 519 679T529 683Q532 683 561 682T645 680Q696 680 723 681T752 682Q767 682 767 672Q767 650 759 642Q756 637 737 637Q666 633 648 597Q646 592 598 404Q557 235 548 205Q515 105 433 42T263 -22Q171 -22 116 34T60 167V183Q60 201 115 421Q164 622 164 628Q164 635 107 637Z"></path><path stroke-width="0" id="E2721-MJMAIN-2190" d="M944 261T944 250T929 230H165Q167 228 182 216T211 189T244 152T277 96T303 25Q308 7 308 0Q308 -11 288 -11Q281 -11 278 -11T272 -7T267 2T263 21Q245 94 195 151T73 236Q58 242 55 247Q55 254 59 257T73 264Q121 283 158 314T215 375T247 434T264 480L267 497Q269 503 270 505T275 509T288 511Q308 511 308 500Q308 493 303 475Q293 438 278 406T246 352T215 315T185 287T165 270H929Q944 261 944 250Z"></path><path stroke-width="0" id="E2721-MJMAIN-2B" d="M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"></path><path stroke-width="0" id="E2721-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2721-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2721-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2721-MJMAIN-34" d="M462 0Q444 3 333 3Q217 3 199 0H190V46H221Q241 46 248 46T265 48T279 53T286 61Q287 63 287 115V165H28V211L179 442Q332 674 334 675Q336 677 355 677H373L379 671V211H471V165H379V114Q379 73 379 66T385 54Q393 47 442 46H471V0H462ZM293 211V545L74 212L183 211H293Z"></path><path stroke-width="0" id="E2721-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2721-MJMATHI-49" d="M43 1Q26 1 26 10Q26 12 29 24Q34 43 39 45Q42 46 54 46H60Q120 46 136 53Q137 53 138 54Q143 56 149 77T198 273Q210 318 216 344Q286 624 286 626Q284 630 284 631Q274 637 213 637H193Q184 643 189 662Q193 677 195 680T209 683H213Q285 681 359 681Q481 681 487 683H497Q504 676 504 672T501 655T494 639Q491 637 471 637Q440 637 407 634Q393 631 388 623Q381 609 337 432Q326 385 315 341Q245 65 245 59Q245 52 255 50T307 46H339Q345 38 345 37T342 19Q338 6 332 0H316Q279 2 179 2Q143 2 113 2T65 2T43 1Z"></path><path stroke-width="0" id="E2721-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2721-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path><path stroke-width="0" id="E2721-MJMAIN-6C" d="M42 46H56Q95 46 103 60V68Q103 77 103 91T103 124T104 167T104 217T104 272T104 329Q104 366 104 407T104 482T104 542T103 586T103 603Q100 622 89 628T44 637H26V660Q26 683 28 683L38 684Q48 685 67 686T104 688Q121 689 141 690T171 693T182 694H185V379Q185 62 186 60Q190 52 198 49Q219 46 247 46H263V0H255L232 1Q209 2 183 2T145 3T107 3T57 1L34 0H26V46H42Z"></path><path stroke-width="0" id="E2721-MJMAIN-6E" d="M41 46H55Q94 46 102 60V68Q102 77 102 91T102 122T103 161T103 203Q103 234 103 269T102 328V351Q99 370 88 376T43 385H25V408Q25 431 27 431L37 432Q47 433 65 434T102 436Q119 437 138 438T167 441T178 442H181V402Q181 364 182 364T187 369T199 384T218 402T247 421T285 437Q305 442 336 442Q450 438 463 329Q464 322 464 190V104Q464 66 466 59T477 49Q498 46 526 46H542V0H534L510 1Q487 2 460 2T422 3Q319 3 310 0H302V46H318Q379 46 379 62Q380 64 380 200Q379 335 378 343Q372 371 358 385T334 402T308 404Q263 404 229 370Q202 343 195 315T187 232V168V108Q187 78 188 68T191 55T200 49Q221 46 249 46H265V0H257L234 1Q210 2 183 2T145 3Q42 3 33 0H25V46H41Z"></path><path stroke-width="0" id="E2721-MJMATHI-3B1" d="M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z"></path><path stroke-width="0" id="E2721-MJMAIN-2207" d="M46 676Q46 679 51 683H781Q786 679 786 676Q786 674 617 326T444 -26Q439 -33 416 -33T388 -26Q385 -22 216 326T46 676ZM697 596Q697 597 445 597T193 596Q195 591 319 336T445 80L697 596Z"></path><path stroke-width="0" id="E2721-MJMAIN-35" d="M164 157Q164 133 148 117T109 101H102Q148 22 224 22Q294 22 326 82Q345 115 345 210Q345 313 318 349Q292 382 260 382H254Q176 382 136 314Q132 307 129 306T114 304Q97 304 95 310Q93 314 93 485V614Q93 664 98 664Q100 666 102 666Q103 666 123 658T178 642T253 634Q324 634 389 662Q397 666 402 666Q410 666 410 648V635Q328 538 205 538Q174 538 149 544L139 546V374Q158 388 169 396T205 412T256 420Q337 420 393 355T449 201Q449 109 385 44T229 -22Q148 -22 99 32T50 154Q50 178 61 192T84 210T107 214Q132 214 148 197T164 157Z"></path><path stroke-width="0" id="E2721-MJMAIN-36" d="M42 313Q42 476 123 571T303 666Q372 666 402 630T432 550Q432 525 418 510T379 495Q356 495 341 509T326 548Q326 592 373 601Q351 623 311 626Q240 626 194 566Q147 500 147 364L148 360Q153 366 156 373Q197 433 263 433H267Q313 433 348 414Q372 400 396 374T435 317Q456 268 456 210V192Q456 169 451 149Q440 90 387 34T253 -22Q225 -22 199 -14T143 16T92 75T56 172T42 313ZM257 397Q227 397 205 380T171 335T154 278T148 216Q148 133 160 97T198 39Q222 21 251 21Q302 21 329 59Q342 77 347 104T352 209Q352 289 347 316T329 361Q302 397 257 397Z"></path><path stroke-width="0" id="E2721-MJMAIN-37" d="M55 458Q56 460 72 567L88 674Q88 676 108 676H128V672Q128 662 143 655T195 646T364 644H485V605L417 512Q408 500 387 472T360 435T339 403T319 367T305 330T292 284T284 230T278 162T275 80Q275 66 275 52T274 28V19Q270 2 255 -10T221 -22Q210 -22 200 -19T179 0T168 40Q168 198 265 368Q285 400 349 489L395 552H302Q128 552 119 546Q113 543 108 522T98 479L95 458V455H55V458Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><g transform="translate(13097,-2531)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">算</text><g transform="translate(1052,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">法</text></g><use transform="scale(1.2)" xlink:href="#E2721-MJMAINB-38" x="1963" y="0"></use><use transform="scale(1.2)" xlink:href="#E2721-MJMAINB-2D" x="2538" y="0"></use><use transform="scale(1.2)" xlink:href="#E2721-MJMAINB-32" x="2921" y="0"></use><g transform="translate(4945,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">优</text></g><g transform="translate(5998,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">势</text></g><g transform="translate(7014,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(8067,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(9120,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">者</text></g><use transform="scale(1.2)" xlink:href="#E2721-MJMAINB-2F" x="8477" y="0"></use><g transform="translate(10862,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">评</text></g><g transform="translate(11915,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">论</text></g><g transform="translate(12968,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">者</text></g><g transform="translate(14021,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(15074,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">法</text></g></g><g transform="translate(0,-12251)"><g transform="translate(-19,0)"><g transform="translate(0,8189)"><g><rect fill="black" stroke="none" width="1569" height="100" x="0" y="500"></rect></g></g><g transform="translate(0,-7990)"><g><rect fill="black" stroke="none" width="1569" height="100" x="0" y="-500"></rect></g></g></g><g transform="translate(1551,0)"><g transform="translate(0,8189)"><g><rect fill="black" stroke="none" width="41597" height="100" x="0" y="500"></rect></g></g><g transform="translate(0,6889)"><use xlink:href="#E2721-MJMAIN-22EF" x="166" y="0"></use><g transform="translate(2505,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">同</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">法</text></g><use xlink:href="#E2721-MJMAIN-38" x="2741" y="0"></use><use xlink:href="#E2721-MJMAIN-2D" x="3241" y="0"></use><use xlink:href="#E2721-MJMAIN-31" x="3574" y="0"></use></g><use xlink:href="#E2721-MJMAIN-22EF" x="7746" y="0"></use></g><g transform="translate(0,5589)"><use xlink:href="#E2721-MJMAIN-32"></use><use xlink:href="#E2721-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2721-MJMAIN-32" x="778" y="0"></use><g transform="translate(1278,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2108,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">决</text></g><g transform="translate(2939,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">定</text></g><g transform="translate(3769,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">初</text></g><g transform="translate(4600,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">始</text></g><g transform="translate(5431,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(6261,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g><g transform="translate(7092,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7923,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">选</text></g><g transform="translate(8753,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">择</text></g><g transform="translate(9584,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(10415,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g><use xlink:href="#E2721-MJMATHI-53" x="11495" y="0"></use><g transform="translate(12140,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g><g transform="translate(0,4289)"><use xlink:href="#E2721-MJMAIN-32"></use><use xlink:href="#E2721-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2721-MJMAIN-33" x="778" y="0"></use><g transform="translate(1972,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">如</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">果</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">回</text></g><g transform="translate(2491,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">合</text></g><g transform="translate(3322,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">未</text></g><g transform="translate(4153,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">结</text></g><g transform="translate(4983,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">束</text></g><g transform="translate(5814,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(6645,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(7475,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(8306,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">以</text></g><g transform="translate(9137,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">下</text></g><g transform="translate(9967,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">操</text></g><g transform="translate(10798,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g><g transform="translate(11629,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">：</text></g></g></g><g transform="translate(0,2989)"><g transform="translate(2000,0)"><use xlink:href="#E2721-MJMAIN-32"></use><use xlink:href="#E2721-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2721-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2721-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2721-MJMAIN-31" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">采</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">样</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">用</text></g><g transform="translate(6459,0)"><use xlink:href="#E2721-MJMATHI-3C0" x="0" y="0"></use><use xlink:href="#E2721-MJMAIN-28" x="573" y="0"></use><use xlink:href="#E2721-MJMAIN-22C5" x="962" y="0"></use><use xlink:href="#E2721-MJMAIN-2223" x="1517" y="0"></use><use xlink:href="#E2721-MJMATHI-53" x="2073" y="0"></use><use xlink:href="#E2721-MJMAIN-3B" x="2718" y="0"></use><use xlink:href="#E2721-MJMATHI-3B8" x="3163" y="0"></use><use xlink:href="#E2721-MJMAIN-29" x="3632" y="0"></use></g><g transform="translate(10480,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">得</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">到</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">动</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g></g><use xlink:href="#E2721-MJMATHI-41" x="14303" y="0"></use><g transform="translate(15053,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,1630)"><g transform="translate(2000,0)"><use xlink:href="#E2721-MJMAIN-32"></use><use xlink:href="#E2721-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2721-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2721-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2721-MJMAIN-32" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">动</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g><use xlink:href="#E2721-MJMATHI-41" x="8951" y="0"></use><g transform="translate(9701,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">得</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">到</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">奖</text></g><g transform="translate(3572,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">励</text></g></g><use xlink:href="#E2721-MJMATHI-52" x="14354" y="0"></use><g transform="translate(15113,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">和</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">观</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">测</text></g></g><g transform="translate(18105,0)"><use xlink:href="#E2721-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2721-MJMAIN-2032" x="925" y="583"></use></g><g transform="translate(19054,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,321)"><g transform="translate(2000,0)"><use xlink:href="#E2721-MJMAIN-32"></use><use xlink:href="#E2721-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2721-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2721-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2721-MJMAIN-33" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">估</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">回</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">报</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><use xlink:href="#E2721-MJMATHI-55" x="0" y="0"></use><use xlink:href="#E2721-MJMAIN-2190" x="1044" y="0"></use><use xlink:href="#E2721-MJMATHI-52" x="2322" y="0"></use><use xlink:href="#E2721-MJMAIN-2B" x="3303" y="0"></use><use xlink:href="#E2721-MJMATHI-3B3" x="4304" y="0"></use><use xlink:href="#E2721-MJMATHI-76" x="4847" y="0"></use><use xlink:href="#E2721-MJMAIN-28" x="5332" y="0"></use><g transform="translate(5721,0)"><use xlink:href="#E2721-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2721-MJMAIN-2032" x="925" y="583"></use></g><use xlink:href="#E2721-MJMAIN-3B" x="6670" y="0"></use><use xlink:href="#E2721-MJMAINB-77" x="7114" y="0"></use><use xlink:href="#E2721-MJMAIN-29" x="7945" y="0"></use></g><g transform="translate(15374,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-1029)"><g transform="translate(2000,0)"><use xlink:href="#E2721-MJMAIN-32"></use><use xlink:href="#E2721-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2721-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2721-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2721-MJMAIN-34" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">略</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">改</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">进</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><use xlink:href="#E2721-MJMATHI-3B8" x="8951" y="0"></use><g transform="translate(9420,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">以</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">减</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">小</text></g></g><g transform="translate(12412,0)"><use xlink:href="#E2721-MJMAIN-2212" x="0" y="0"></use><use xlink:href="#E2721-MJMATHI-49" x="778" y="0"></use><use xlink:href="#E2721-MJMAIN-5B" x="1282" y="0"></use><use xlink:href="#E2721-MJMATHI-55" x="1560" y="0"></use><use xlink:href="#E2721-MJMAIN-2212" x="2549" y="0"></use><use xlink:href="#E2721-MJMATHI-76" x="3549" y="0"></use><use xlink:href="#E2721-MJMAIN-28" x="4034" y="0"></use><use xlink:href="#E2721-MJMATHI-53" x="4423" y="0"></use><use xlink:href="#E2721-MJMAIN-3B" x="5068" y="0"></use><use xlink:href="#E2721-MJMAINB-77" x="5513" y="0"></use><use xlink:href="#E2721-MJMAIN-29" x="6344" y="0"></use><use xlink:href="#E2721-MJMAIN-5D" x="6733" y="0"></use><g transform="translate(7177,0)"><use xlink:href="#E2721-MJMAIN-6C"></use><use xlink:href="#E2721-MJMAIN-6E" x="278" y="0"></use></g><use xlink:href="#E2721-MJMATHI-3C0" x="8178" y="0"></use><use xlink:href="#E2721-MJMAIN-28" x="8751" y="0"></use><use xlink:href="#E2721-MJMATHI-41" x="9140" y="0"></use><use xlink:href="#E2721-MJMAIN-2223" x="10168" y="0"></use><use xlink:href="#E2721-MJMATHI-53" x="10724" y="0"></use><use xlink:href="#E2721-MJMAIN-3B" x="11369" y="0"></use><use xlink:href="#E2721-MJMATHI-3B8" x="11813" y="0"></use><use xlink:href="#E2721-MJMAIN-29" x="12282" y="0"></use></g><g transform="translate(25083,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g></g></g></g><g transform="translate(0,-2522)"><g transform="translate(4444,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">如</text><g transform="translate(1080,0)"><use xlink:href="#E2721-MJMATHI-3B8" x="0" y="0"></use><use xlink:href="#E2721-MJMAIN-2190" x="746" y="0"></use><use xlink:href="#E2721-MJMATHI-3B8" x="2024" y="0"></use><use xlink:href="#E2721-MJMAIN-2B" x="2715" y="0"></use><g transform="translate(3716,0)"><use xlink:href="#E2721-MJMATHI-3B1" x="0" y="0"></use><g transform="translate(640,412)"><use transform="scale(0.707)" xlink:href="#E2721-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2721-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2721-MJMAIN-29" x="858" y="0"></use></g></g><use xlink:href="#E2721-MJMATHI-49" x="5337" y="0"></use><use xlink:href="#E2721-MJMAIN-5B" x="5841" y="0"></use><use xlink:href="#E2721-MJMATHI-55" x="6119" y="0"></use><use xlink:href="#E2721-MJMAIN-2212" x="7108" y="0"></use><use xlink:href="#E2721-MJMATHI-76" x="8109" y="0"></use><use xlink:href="#E2721-MJMAIN-28" x="8594" y="0"></use><use xlink:href="#E2721-MJMATHI-53" x="8983" y="0"></use><use xlink:href="#E2721-MJMAIN-3B" x="9628" y="0"></use><use xlink:href="#E2721-MJMAINB-77" x="10072" y="0"></use><use xlink:href="#E2721-MJMAIN-29" x="10903" y="0"></use><use xlink:href="#E2721-MJMAIN-5D" x="11292" y="0"></use><use xlink:href="#E2721-MJMAIN-2207" x="11570" y="0"></use><g transform="translate(12570,0)"><use xlink:href="#E2721-MJMAIN-6C"></use><use xlink:href="#E2721-MJMAIN-6E" x="278" y="0"></use></g><use xlink:href="#E2721-MJMATHI-3C0" x="13571" y="0"></use><use xlink:href="#E2721-MJMAIN-28" x="14144" y="0"></use><use xlink:href="#E2721-MJMATHI-41" x="14533" y="0"></use><use xlink:href="#E2721-MJMAIN-2223" x="15560" y="0"></use><use xlink:href="#E2721-MJMATHI-53" x="16116" y="0"></use><use xlink:href="#E2721-MJMAIN-3B" x="16761" y="0"></use><use xlink:href="#E2721-MJMATHI-3B8" x="17206" y="0"></use><use xlink:href="#E2721-MJMAIN-29" x="17675" y="0"></use></g><g transform="translate(19145,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-4015)"><g transform="translate(2000,0)"><use xlink:href="#E2721-MJMAIN-32"></use><use xlink:href="#E2721-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2721-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2721-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2721-MJMAIN-35" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">价</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">值</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><use xlink:href="#E2721-MJMAINB-77" x="8951" y="0"></use><g transform="translate(9782,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">以</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">减</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">小</text></g></g><g transform="translate(12774,0)"><use xlink:href="#E2721-MJMAIN-5B" x="0" y="0"></use><use xlink:href="#E2721-MJMATHI-55" x="278" y="0"></use><use xlink:href="#E2721-MJMAIN-2212" x="1267" y="0"></use><use xlink:href="#E2721-MJMATHI-76" x="2267" y="0"></use><use xlink:href="#E2721-MJMAIN-28" x="2752" y="0"></use><use xlink:href="#E2721-MJMATHI-53" x="3141" y="0"></use><use xlink:href="#E2721-MJMAIN-3B" x="3786" y="0"></use><use xlink:href="#E2721-MJMAINB-77" x="4231" y="0"></use><use xlink:href="#E2721-MJMAIN-29" x="5062" y="0"></use><g transform="translate(5451,0)"><use xlink:href="#E2721-MJMAIN-5D" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2721-MJMAIN-32" x="393" y="583"></use></g></g><g transform="translate(18956,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">如</text></g></g><g transform="translate(21118,0)"><use xlink:href="#E2721-MJMAINB-77" x="0" y="0"></use><use xlink:href="#E2721-MJMAIN-2190" x="1108" y="0"></use><use xlink:href="#E2721-MJMAINB-77" x="2386" y="0"></use><use xlink:href="#E2721-MJMAIN-2B" x="3439" y="0"></use><g transform="translate(4440,0)"><use xlink:href="#E2721-MJMATHI-3B1" x="0" y="0"></use><g transform="translate(640,412)"><use transform="scale(0.707)" xlink:href="#E2721-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2721-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2721-MJMAIN-29" x="1220" y="0"></use></g></g><use xlink:href="#E2721-MJMAIN-5B" x="6317" y="0"></use><use xlink:href="#E2721-MJMATHI-55" x="6595" y="0"></use><use xlink:href="#E2721-MJMAIN-2212" x="7584" y="0"></use><use xlink:href="#E2721-MJMATHI-76" x="8585" y="0"></use><use xlink:href="#E2721-MJMAIN-28" x="9070" y="0"></use><use xlink:href="#E2721-MJMATHI-53" x="9459" y="0"></use><use xlink:href="#E2721-MJMAIN-3B" x="10104" y="0"></use><use xlink:href="#E2721-MJMAINB-77" x="10548" y="0"></use><use xlink:href="#E2721-MJMAIN-29" x="11379" y="0"></use><use xlink:href="#E2721-MJMAIN-5D" x="11768" y="0"></use><use xlink:href="#E2721-MJMAIN-2207" x="12046" y="0"></use><use xlink:href="#E2721-MJMATHI-76" x="12879" y="0"></use><use xlink:href="#E2721-MJMAIN-28" x="13364" y="0"></use><use xlink:href="#E2721-MJMATHI-53" x="13753" y="0"></use><use xlink:href="#E2721-MJMAIN-3B" x="14398" y="0"></use><use xlink:href="#E2721-MJMAINB-77" x="14843" y="0"></use><use xlink:href="#E2721-MJMAIN-29" x="15674" y="0"></use></g><g transform="translate(37181,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-5365)"><g transform="translate(2000,0)"><use xlink:href="#E2721-MJMAIN-32"></use><use xlink:href="#E2721-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2721-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2721-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2721-MJMAIN-36" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">累</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">折</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">扣</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(8701,0)"><use xlink:href="#E2721-MJMATHI-49" x="0" y="0"></use><use xlink:href="#E2721-MJMAIN-2190" x="781" y="0"></use><use xlink:href="#E2721-MJMATHI-3B3" x="2059" y="0"></use><use xlink:href="#E2721-MJMATHI-49" x="2602" y="0"></use></g><g transform="translate(11807,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-6690)"><g transform="translate(2000,0)"><use xlink:href="#E2721-MJMAIN-32"></use><use xlink:href="#E2721-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2721-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2721-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2721-MJMAIN-37" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><use xlink:href="#E2721-MJMATHI-53" x="0" y="0"></use><use xlink:href="#E2721-MJMAIN-2190" x="922" y="0"></use><g transform="translate(2200,0)"><use xlink:href="#E2721-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2721-MJMAIN-2032" x="925" y="583"></use></g></g><g transform="translate(10189,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g></g><g transform="translate(0,-7990)"><g><rect fill="black" stroke="none" width="41597" height="100" x="0" y="-500"></rect></g></g></g></g></g></svg></span></div><script type="math/tex; mode=display" id="MathJax-Element-1988">\; \\ \; \\
\large \textbf{算法 8-2   优势执行者/评论者算法} \\
\begin{split}
\rule[5pt]{10mm}{0.1em} &\rule[5pt]{265mm}{0.1em} \\
&\cdots \quad \text{同算法 8-1} \quad \cdots \\
&\text{2.2（决定初始状态）选择状态 $S$ 。} \\
&\text{2.3 $\;\,$如果回合未结束，执行以下操作：} \\
&\qquad \text{2.3.1（采样）用 $\pi(\cdot \mid S; \theta)$ 得到动作 $A$ ；} \\
&\qquad \text{2.3.2（执行）执行动作 $A$ ，得到奖励 $R$ 和观测 $S'$ ；} \\
&\qquad \text{2.3.3（估计回报）$U \leftarrow R + \gamma v(S';\bold w)$ ；} \\
&\qquad \text{2.3.4（策略改进）更新 $\theta$ 以减小 $-I[U-v(S;\bold w)] \ln \pi(A \mid S; \theta)$ ，} \\
&\qquad \qquad \;\, \text{如 $\theta \leftarrow \theta + \alpha^{(\theta)} I[U-v(S;\bold w)] \nabla \ln \pi(A \mid S; \theta)$ ；} \\
&\qquad \text{2.3.5（更新价值）更新 $\bold w$ 以减小 $[U - v(S;\bold w)]^2$ ，如 $\bold w \leftarrow \bold w + \alpha^{(\bold w)} [U - v(S;\bold w)] \nabla v(S;\bold w)$ ；} \\
&\qquad \text{2.3.6（更新累计折扣）$I \leftarrow \gamma I$ ；} \\
&\qquad \text{2.3.7（更新状态）$S \leftarrow S'$ 。} \\
\rule[-5pt]{10mm}{0.1em} &\rule[-5pt]{265mm}{0.1em}
\end{split}
\; \\ \; \\</script>
					</div></div><div contenteditable="false" spellcheck="false" class="mathjax-block md-end-block md-math-block md-rawblock" id="mathjax-n19" cid="n19" mdtype="math_block"><div class="md-rawblock-container md-math-container" tabindex="-1"><div class="MathJax_SVG_Display"><span class="MathJax_SVG" id="MathJax-Element-1227-Frame" tabindex="-1" style="font-size: 100%; display: inline-block; zoom: 0.971345;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="101.294ex" height="62.131ex" viewBox="-18.1 -43.5 43612.5 26751" role="img" focusable="false" style="vertical-align: -62.03ex; margin-left: -0.042ex; max-width: 100%;"><defs><path stroke-width="0" id="E2683-MJMAINB-38" d="M80 474Q80 561 139 607T278 654Q357 654 411 632Q490 593 494 509Q494 424 416 376L407 371L418 364Q432 356 447 345T481 312T513 260T526 192Q526 100 461 45T285 -11Q184 -11 116 32T48 164Q48 181 50 196T58 225T69 249T84 270T100 286T117 300T134 311T149 321T162 329L152 336Q120 360 100 397T80 474ZM347 404Q404 446 404 503Q404 579 317 599Q309 600 276 600Q178 600 170 538Q170 532 171 527T173 518T178 509T184 501T194 492T205 484T219 476T235 467T254 456T275 445L347 404ZM289 47Q323 47 351 54T402 82T425 137Q425 147 421 161Q411 183 391 197T303 249Q224 293 223 293Q220 291 215 288T197 273T175 248T157 213T149 167Q149 109 188 78T289 47Z"></path><path stroke-width="0" id="E2683-MJMAINB-2D" d="M13 166V278H318V166H13Z"></path><path stroke-width="0" id="E2683-MJMAINB-33" d="M80 503Q80 565 133 610T274 655Q366 655 421 623T491 538Q493 528 493 510Q493 446 453 407T361 348L376 344Q452 324 489 281T526 184Q526 152 514 121T474 58T392 8T265 -11Q175 -11 111 34T48 152Q50 187 72 209T132 232Q171 232 193 208T216 147Q216 136 214 126T207 108T197 94T187 84T178 77T170 72L168 71Q168 70 179 65T215 54T266 48H270Q331 48 350 105Q358 128 358 185Q358 239 348 268T309 313Q292 321 242 322Q205 322 198 324T191 341V348Q191 366 196 369T232 375Q239 375 247 376T260 377T268 378Q284 383 297 393T326 436T341 517Q341 536 339 547T331 573T308 593T266 600Q248 600 241 599Q214 593 183 576Q234 556 234 503Q234 462 210 444T157 426Q126 426 103 446T80 503Z"></path><path stroke-width="0" id="E2683-MJMAINB-2F" d="M451 730Q460 750 479 750Q492 750 501 740T510 718Q508 708 318 244L122 -232Q112 -250 95 -250Q82 -250 73 -241T64 -218Q66 -205 258 261T451 730Z"></path><path stroke-width="0" id="E2683-MJMAIN-22EF" d="M78 250Q78 274 95 292T138 310Q162 310 180 294T199 251Q199 226 182 208T139 190T96 207T78 250ZM525 250Q525 274 542 292T585 310Q609 310 627 294T646 251Q646 226 629 208T586 190T543 207T525 250ZM972 250Q972 274 989 292T1032 310Q1056 310 1074 294T1093 251Q1093 226 1076 208T1033 190T990 207T972 250Z"></path><path stroke-width="0" id="E2683-MJMAIN-38" d="M70 417T70 494T124 618T248 666Q319 666 374 624T429 515Q429 485 418 459T392 417T361 389T335 371T324 363L338 354Q352 344 366 334T382 323Q457 264 457 174Q457 95 399 37T249 -22Q159 -22 101 29T43 155Q43 263 172 335L154 348Q133 361 127 368Q70 417 70 494ZM286 386L292 390Q298 394 301 396T311 403T323 413T334 425T345 438T355 454T364 471T369 491T371 513Q371 556 342 586T275 624Q268 625 242 625Q201 625 165 599T128 534Q128 511 141 492T167 463T217 431Q224 426 228 424L286 386ZM250 21Q308 21 350 55T392 137Q392 154 387 169T375 194T353 216T330 234T301 253T274 270Q260 279 244 289T218 306L210 311Q204 311 181 294T133 239T107 157Q107 98 150 60T250 21Z"></path><path stroke-width="0" id="E2683-MJMAIN-2D" d="M11 179V252H277V179H11Z"></path><path stroke-width="0" id="E2683-MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="0" id="E2683-MJMAIN-2E" d="M78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60Z"></path><path stroke-width="0" id="E2683-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path><path stroke-width="0" id="E2683-MJMAIN-2032" d="M79 43Q73 43 52 49T30 61Q30 68 85 293T146 528Q161 560 198 560Q218 560 240 545T262 501Q262 496 260 486Q259 479 173 263T84 45T79 43Z"></path><path stroke-width="0" id="E2683-MJMAIN-2190" d="M944 261T944 250T929 230H165Q167 228 182 216T211 189T244 152T277 96T303 25Q308 7 308 0Q308 -11 288 -11Q281 -11 278 -11T272 -7T267 2T263 21Q245 94 195 151T73 236Q58 242 55 247Q55 254 59 257T73 264Q121 283 158 314T215 375T247 434T264 480L267 497Q269 503 270 505T275 509T288 511Q308 511 308 500Q308 493 303 475Q293 438 278 406T246 352T215 315T185 287T165 270H929Q944 261 944 250Z"></path><path stroke-width="0" id="E2683-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2683-MJMAIN-32" d="M109 429Q82 429 66 447T50 491Q50 562 103 614T235 666Q326 666 387 610T449 465Q449 422 429 383T381 315T301 241Q265 210 201 149L142 93L218 92Q375 92 385 97Q392 99 409 186V189H449V186Q448 183 436 95T421 3V0H50V19V31Q50 38 56 46T86 81Q115 113 136 137Q145 147 170 174T204 211T233 244T261 278T284 308T305 340T320 369T333 401T340 431T343 464Q343 527 309 573T212 619Q179 619 154 602T119 569T109 550Q109 549 114 549Q132 549 151 535T170 489Q170 464 154 447T109 429Z"></path><path stroke-width="0" id="E2683-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2683-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2683-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2683-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2683-MJMAIN-30" d="M96 585Q152 666 249 666Q297 666 345 640T423 548Q460 465 460 320Q460 165 417 83Q397 41 362 16T301 -15T250 -22Q224 -22 198 -16T137 16T82 83Q39 165 39 320Q39 494 96 585ZM321 597Q291 629 250 629Q208 629 178 597Q153 571 145 525T137 333Q137 175 145 125T181 46Q209 16 250 16Q290 16 318 46Q347 76 354 130T362 333Q362 478 354 524T321 597Z"></path><path stroke-width="0" id="E2683-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2683-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2683-MJMATHI-52" d="M230 637Q203 637 198 638T193 649Q193 676 204 682Q206 683 378 683Q550 682 564 680Q620 672 658 652T712 606T733 563T739 529Q739 484 710 445T643 385T576 351T538 338L545 333Q612 295 612 223Q612 212 607 162T602 80V71Q602 53 603 43T614 25T640 16Q668 16 686 38T712 85Q717 99 720 102T735 105Q755 105 755 93Q755 75 731 36Q693 -21 641 -21H632Q571 -21 531 4T487 82Q487 109 502 166T517 239Q517 290 474 313Q459 320 449 321T378 323H309L277 193Q244 61 244 59Q244 55 245 54T252 50T269 48T302 46H333Q339 38 339 37T336 19Q332 6 326 0H311Q275 2 180 2Q146 2 117 2T71 2T50 1Q33 1 33 10Q33 12 36 24Q41 43 46 45Q50 46 61 46H67Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628Q287 635 230 637ZM630 554Q630 586 609 608T523 636Q521 636 500 636T462 637H440Q393 637 386 627Q385 624 352 494T319 361Q319 360 388 360Q466 361 492 367Q556 377 592 426Q608 449 619 486T630 554Z"></path><path stroke-width="0" id="E2683-MJMATHI-54" d="M40 437Q21 437 21 445Q21 450 37 501T71 602L88 651Q93 669 101 677H569H659Q691 677 697 676T704 667Q704 661 687 553T668 444Q668 437 649 437Q640 437 637 437T631 442L629 445Q629 451 635 490T641 551Q641 586 628 604T573 629Q568 630 515 631Q469 631 457 630T439 622Q438 621 368 343T298 60Q298 48 386 46Q418 46 427 45T436 36Q436 31 433 22Q429 4 424 1L422 0Q419 0 415 0Q410 0 363 1T228 2Q99 2 64 0H49Q43 6 43 9T45 27Q49 40 55 46H83H94Q174 46 189 55Q190 56 191 56Q196 59 201 76T241 233Q258 301 269 344Q339 619 339 625Q339 630 310 630H279Q212 630 191 624Q146 614 121 583T67 467Q60 445 57 441T43 437H40Z"></path><path stroke-width="0" id="E2683-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2683-MJMATHI-55" d="M107 637Q73 637 71 641Q70 643 70 649Q70 673 81 682Q83 683 98 683Q139 681 234 681Q268 681 297 681T342 682T362 682Q378 682 378 672Q378 670 376 658Q371 641 366 638H364Q362 638 359 638T352 638T343 637T334 637Q295 636 284 634T266 623Q265 621 238 518T184 302T154 169Q152 155 152 140Q152 86 183 55T269 24Q336 24 403 69T501 205L552 406Q599 598 599 606Q599 633 535 637Q511 637 511 648Q511 650 513 660Q517 676 519 679T529 683Q532 683 561 682T645 680Q696 680 723 681T752 682Q767 682 767 672Q767 650 759 642Q756 637 737 637Q666 633 648 597Q646 592 598 404Q557 235 548 205Q515 105 433 42T263 -22Q171 -22 116 34T60 167V183Q60 201 115 421Q164 622 164 628Q164 635 107 637Z"></path><path stroke-width="0" id="E2683-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2683-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2683-MJMAINB-67" d="M50 300Q50 368 105 409T255 450Q328 450 376 426L388 420Q435 455 489 455Q517 455 533 441T554 414T558 389Q558 367 544 353T508 339Q484 339 471 354T458 387Q458 397 462 400Q464 401 461 400Q459 400 454 399Q429 392 427 390Q454 353 459 328Q461 315 461 300Q461 240 419 202Q364 149 248 149Q185 149 136 172Q129 158 129 148Q129 105 170 93Q176 91 263 91Q273 91 298 91T334 91T366 89T400 85T432 77T466 64Q544 22 544 -69Q544 -114 506 -145Q438 -201 287 -201Q149 -201 90 -161T30 -70Q30 -58 33 -47T42 -27T54 -13T69 -1T82 6T94 12T101 15Q66 57 66 106Q66 151 90 187L97 197L89 204Q50 243 50 300ZM485 403H492Q491 404 488 404L485 403V403ZM255 200Q279 200 295 206T319 219T331 242T335 268T336 300Q336 337 333 352T317 380Q298 399 255 399Q228 399 211 392T187 371T178 345T176 312V300V289Q176 235 194 219Q215 200 255 200ZM287 -150Q357 -150 400 -128T443 -71Q443 -65 442 -61T436 -50T420 -37T389 -27T339 -21L308 -20Q276 -20 253 -20Q190 -20 180 -20T156 -26Q130 -38 130 -69Q130 -105 173 -127T287 -150Z"></path><path stroke-width="0" id="E2683-MJMAIN-33" d="M127 463Q100 463 85 480T69 524Q69 579 117 622T233 665Q268 665 277 664Q351 652 390 611T430 522Q430 470 396 421T302 350L299 348Q299 347 308 345T337 336T375 315Q457 262 457 175Q457 96 395 37T238 -22Q158 -22 100 21T42 130Q42 158 60 175T105 193Q133 193 151 175T169 130Q169 119 166 110T159 94T148 82T136 74T126 70T118 67L114 66Q165 21 238 21Q293 21 321 74Q338 107 338 175V195Q338 290 274 322Q259 328 213 329L171 330L168 332Q166 335 166 348Q166 366 174 366Q202 366 232 371Q266 376 294 413T322 525V533Q322 590 287 612Q265 626 240 626Q208 626 181 615T143 592T132 580H135Q138 579 143 578T153 573T165 566T175 555T183 540T186 520Q186 498 172 481T127 463Z"></path><path stroke-width="0" id="E2683-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2683-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path stroke-width="0" id="E2683-MJMAIN-2B" d="M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"></path><path stroke-width="0" id="E2683-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2683-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2683-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path><path stroke-width="0" id="E2683-MJMAIN-2207" d="M46 676Q46 679 51 683H781Q786 679 786 676Q786 674 617 326T444 -26Q439 -33 416 -33T388 -26Q385 -22 216 326T46 676ZM697 596Q697 597 445 597T193 596Q195 591 319 336T445 80L697 596Z"></path><path stroke-width="0" id="E2683-MJMAIN-6C" d="M42 46H56Q95 46 103 60V68Q103 77 103 91T103 124T104 167T104 217T104 272T104 329Q104 366 104 407T104 482T104 542T103 586T103 603Q100 622 89 628T44 637H26V660Q26 683 28 683L38 684Q48 685 67 686T104 688Q121 689 141 690T171 693T182 694H185V379Q185 62 186 60Q190 52 198 49Q219 46 247 46H263V0H255L232 1Q209 2 183 2T145 3T107 3T57 1L34 0H26V46H42Z"></path><path stroke-width="0" id="E2683-MJMAIN-6E" d="M41 46H55Q94 46 102 60V68Q102 77 102 91T102 122T103 161T103 203Q103 234 103 269T102 328V351Q99 370 88 376T43 385H25V408Q25 431 27 431L37 432Q47 433 65 434T102 436Q119 437 138 438T167 441T178 442H181V402Q181 364 182 364T187 369T199 384T218 402T247 421T285 437Q305 442 336 442Q450 438 463 329Q464 322 464 190V104Q464 66 466 59T477 49Q498 46 526 46H542V0H534L510 1Q487 2 460 2T422 3Q319 3 310 0H302V46H318Q379 46 379 62Q380 64 380 200Q379 335 378 343Q372 371 358 385T334 402T308 404Q263 404 229 370Q202 343 195 315T187 232V168V108Q187 78 188 68T191 55T200 49Q221 46 249 46H265V0H257L234 1Q210 2 183 2T145 3Q42 3 33 0H25V46H41Z"></path><path stroke-width="0" id="E2683-MJMAIN-2223" d="M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z"></path><path stroke-width="0" id="E2683-MJMAIN-35" d="M164 157Q164 133 148 117T109 101H102Q148 22 224 22Q294 22 326 82Q345 115 345 210Q345 313 318 349Q292 382 260 382H254Q176 382 136 314Q132 307 129 306T114 304Q97 304 95 310Q93 314 93 485V614Q93 664 98 664Q100 666 102 666Q103 666 123 658T178 642T253 634Q324 634 389 662Q397 666 402 666Q410 666 410 648V635Q328 538 205 538Q174 538 149 544L139 546V374Q158 388 169 396T205 412T256 420Q337 420 393 355T449 201Q449 109 385 44T229 -22Q148 -22 99 32T50 154Q50 178 61 192T84 210T107 214Q132 214 148 197T164 157Z"></path><path stroke-width="0" id="E2683-MJMATHI-3B1" d="M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><g transform="translate(6326,-2531)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">算</text><g transform="translate(1052,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">法</text></g><use transform="scale(1.2)" xlink:href="#E2683-MJMAINB-38" x="1963" y="0"></use><use transform="scale(1.2)" xlink:href="#E2683-MJMAINB-2D" x="2538" y="0"></use><use transform="scale(1.2)" xlink:href="#E2683-MJMAINB-33" x="2921" y="0"></use><g transform="translate(4945,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">异</text></g><g transform="translate(5998,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">步</text></g><g transform="translate(7050,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">优</text></g><g transform="translate(8103,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">势</text></g><g transform="translate(9120,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(10172,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(11225,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">者</text></g><use transform="scale(1.2)" xlink:href="#E2683-MJMAINB-2F" x="10232" y="0"></use><g transform="translate(12968,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">评</text></g><g transform="translate(14021,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">论</text></g><g transform="translate(15074,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">者</text></g><g transform="translate(16127,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(17180,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">法</text></g><g transform="translate(18232,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(19249,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">演</text></g><g transform="translate(20301,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">示</text></g><g transform="translate(21354,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">某</text></g><g transform="translate(22407,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">个</text></g><g transform="translate(23460,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">线</text></g><g transform="translate(24476,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">程</text></g><g transform="translate(25529,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">的</text></g><g transform="translate(26545,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(27598,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">为</text></g><g transform="translate(28651,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">）</text></g></g><g transform="translate(0,-14392)"><g transform="translate(-19,0)"><g transform="translate(0,10330)"><g><rect fill="black" stroke="none" width="1569" height="100" x="0" y="500"></rect></g></g><g transform="translate(0,-10131)"><g><rect fill="black" stroke="none" width="1569" height="100" x="0" y="-500"></rect></g></g></g><g transform="translate(1551,0)"><g transform="translate(0,10330)"><g><rect fill="black" stroke="none" width="41597" height="100" x="0" y="500"></rect></g></g><g transform="translate(0,9030)"><use xlink:href="#E2683-MJMAIN-22EF" x="166" y="0"></use><g transform="translate(2505,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">同</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">法</text></g><use xlink:href="#E2683-MJMAIN-38" x="2741" y="0"></use><use xlink:href="#E2683-MJMAIN-2D" x="3241" y="0"></use><use xlink:href="#E2683-MJMAIN-31" x="3574" y="0"></use></g><use xlink:href="#E2683-MJMAIN-22EF" x="7746" y="0"></use></g><g transform="translate(0,7721)"><use xlink:href="#E2683-MJMAIN-31"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><g transform="translate(778,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(1608,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">同</text></g><g transform="translate(2439,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">步</text></g><g transform="translate(3269,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">全</text></g><g transform="translate(4100,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">局</text></g><g transform="translate(4931,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">参</text></g><g transform="translate(5761,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">数</text></g><g transform="translate(6592,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7423,0)"><use xlink:href="#E2683-MJMATHI-3B8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-2032" x="663" y="583"></use><use xlink:href="#E2683-MJMAIN-2190" x="1041" y="0"></use><use xlink:href="#E2683-MJMATHI-3B8" x="2319" y="0"></use></g><g transform="translate(10211,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g></g><g transform="translate(11291,0)"><use xlink:href="#E2683-MJMAINB-77" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-2032" x="1175" y="583"></use><use xlink:href="#E2683-MJMAIN-2190" x="1403" y="0"></use><use xlink:href="#E2683-MJMAINB-77" x="2681" y="0"></use></g><g transform="translate(14803,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g><g transform="translate(0,6421)"><use xlink:href="#E2683-MJMAIN-32"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><g transform="translate(1472,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">逐</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">回</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">合</text></g><g transform="translate(2491,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(3322,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(4153,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">以</text></g><g transform="translate(4983,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">下</text></g><g transform="translate(5814,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">过</text></g><g transform="translate(6645,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">程</text></g><g transform="translate(7475,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">：</text></g></g></g><g transform="translate(0,5113)"><g transform="translate(2000,0)"><use xlink:href="#E2683-MJMAIN-32"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2683-MJMAIN-31" x="778" y="0"></use><g transform="translate(1972,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">用</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">略</text></g></g><g transform="translate(4714,0)"><use xlink:href="#E2683-MJMATHI-3C0" x="0" y="0"></use><use xlink:href="#E2683-MJMAIN-28" x="573" y="0"></use><g transform="translate(962,0)"><use xlink:href="#E2683-MJMATHI-3B8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-2032" x="663" y="583"></use></g><use xlink:href="#E2683-MJMAIN-29" x="1725" y="0"></use></g><g transform="translate(6828,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">生</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">成</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">轨</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">迹</text></g></g><g transform="translate(10651,0)"><use xlink:href="#E2683-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-30" x="866" y="-213"></use><use xlink:href="#E2683-MJMAIN-2C" x="1066" y="0"></use><g transform="translate(1511,0)"><use xlink:href="#E2683-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-30" x="1060" y="-213"></use></g><use xlink:href="#E2683-MJMAIN-2C" x="2714" y="0"></use><g transform="translate(3159,0)"><use xlink:href="#E2683-MJMATHI-52" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-31" x="1073" y="-213"></use></g><use xlink:href="#E2683-MJMAIN-2C" x="4371" y="0"></use><g transform="translate(4816,0)"><use xlink:href="#E2683-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-31" x="866" y="-213"></use></g><use xlink:href="#E2683-MJMAIN-2C" x="5883" y="0"></use><use xlink:href="#E2683-MJMAIN-22EF" x="6327" y="0"></use><use xlink:href="#E2683-MJMAIN-2C" x="7666" y="0"></use><g transform="translate(8111,0)"><use xlink:href="#E2683-MJMATHI-53" x="0" y="0"></use><g transform="translate(613,-150)"><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-54" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-2212" x="704" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-31" x="1482" y="0"></use></g></g><use xlink:href="#E2683-MJMAIN-2C" x="10225" y="0"></use><g transform="translate(10670,0)"><use xlink:href="#E2683-MJMATHI-41" x="0" y="0"></use><g transform="translate(750,-150)"><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-54" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-2212" x="704" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-31" x="1482" y="0"></use></g></g><use xlink:href="#E2683-MJMAIN-2C" x="12921" y="0"></use><g transform="translate(13366,0)"><use xlink:href="#E2683-MJMATHI-52" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-54" x="1073" y="-213"></use></g><use xlink:href="#E2683-MJMAIN-2C" x="14723" y="0"></use><g transform="translate(15167,0)"><use xlink:href="#E2683-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-54" x="866" y="-213"></use></g></g><g transform="translate(27030,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">直</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">到</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">回</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">合</text></g><g transform="translate(3572,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">结</text></g><g transform="translate(4403,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">束</text></g><g transform="translate(5233,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">或</text></g><g transform="translate(6064,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">达</text></g><g transform="translate(6895,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">步</text></g><g transform="translate(7725,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">数</text></g><g transform="translate(8556,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">上</text></g><g transform="translate(9387,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">限</text></g><g transform="translate(10217,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g></g><g transform="translate(0,3763)"><g transform="translate(2000,0)"><use xlink:href="#E2683-MJMAIN-32"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2683-MJMAIN-32" x="778" y="0"></use><g transform="translate(1972,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">为</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">梯</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">度</text></g><g transform="translate(2491,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(3322,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(4153,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">初</text></g><g transform="translate(4983,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">始</text></g><g transform="translate(5814,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">化</text></g><g transform="translate(6645,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">：</text></g></g></g></g><g transform="translate(0,2454)"><g transform="translate(4000,0)"><use xlink:href="#E2683-MJMAIN-32"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2683-MJMAIN-32" x="778" y="0"></use><use xlink:href="#E2683-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2683-MJMAIN-31" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">初</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">始</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">化</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">目</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">标</text></g><g transform="translate(7289,0)"><use xlink:href="#E2683-MJMATHI-55" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-54" x="965" y="-213"></use></g><g transform="translate(8570,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">若</text></g></g><g transform="translate(10482,0)"><use xlink:href="#E2683-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-54" x="866" y="-213"></use></g><g transform="translate(11692,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">是</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">终</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">止</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(3572,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g><g transform="translate(4403,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(5233,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">则</text></g></g><g transform="translate(18007,0)"><use xlink:href="#E2683-MJMATHI-55" x="0" y="0"></use><use xlink:href="#E2683-MJMAIN-2190" x="1044" y="0"></use><use xlink:href="#E2683-MJMAIN-30" x="2322" y="0"></use></g><g transform="translate(20829,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">否</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">则</text></g></g><g transform="translate(23821,0)"><use xlink:href="#E2683-MJMATHI-55" x="0" y="0"></use><use xlink:href="#E2683-MJMAIN-2190" x="1044" y="0"></use><use xlink:href="#E2683-MJMATHI-76" x="2322" y="0"></use><use xlink:href="#E2683-MJMAIN-28" x="2807" y="0"></use><g transform="translate(3196,0)"><use xlink:href="#E2683-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-54" x="866" y="-213"></use></g><use xlink:href="#E2683-MJMAIN-3B" x="4407" y="0"></use><g transform="translate(4852,0)"><use xlink:href="#E2683-MJMAINB-77" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-2032" x="1175" y="583"></use></g><use xlink:href="#E2683-MJMAIN-29" x="5977" y="0"></use></g><g transform="translate(30188,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,961)"><g transform="translate(4000,0)"><use xlink:href="#E2683-MJMAIN-32"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2683-MJMAIN-32" x="778" y="0"></use><use xlink:href="#E2683-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2683-MJMAIN-32" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">初</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">始</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">化</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">梯</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">度</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(8120,0)"><use xlink:href="#E2683-MJMAINB-67" x="0" y="0"></use><g transform="translate(575,412)"><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-29" x="858" y="0"></use></g><use xlink:href="#E2683-MJMAIN-2190" x="1834" y="0"></use><use xlink:href="#E2683-MJMAIN-30" x="3112" y="0"></use></g><g transform="translate(11732,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g></g><g transform="translate(12813,0)"><use xlink:href="#E2683-MJMAINB-67" x="0" y="0"></use><g transform="translate(575,412)"><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-29" x="1220" y="0"></use></g><use xlink:href="#E2683-MJMAIN-2190" x="2090" y="0"></use><use xlink:href="#E2683-MJMAIN-30" x="3368" y="0"></use></g><g transform="translate(16681,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g></g><g transform="translate(0,-340)"><g transform="translate(2000,0)"><use xlink:href="#E2683-MJMAIN-32"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2683-MJMAIN-33" x="778" y="0"></use><g transform="translate(1278,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2108,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">异</text></g><g transform="translate(2939,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">步</text></g><g transform="translate(3769,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(4600,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(5431,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">梯</text></g><g transform="translate(6261,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">度</text></g><g transform="translate(7092,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7923,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">对</text></g><g transform="translate(9003,0)"><use xlink:href="#E2683-MJMATHI-74" x="0" y="0"></use><use xlink:href="#E2683-MJMAIN-3D" x="638" y="0"></use><use xlink:href="#E2683-MJMATHI-54" x="1694" y="0"></use><use xlink:href="#E2683-MJMAIN-2212" x="2620" y="0"></use><use xlink:href="#E2683-MJMAIN-31" x="3621" y="0"></use><use xlink:href="#E2683-MJMAIN-2C" x="4121" y="0"></use><use xlink:href="#E2683-MJMATHI-54" x="4565" y="0"></use><use xlink:href="#E2683-MJMAIN-2212" x="5491" y="0"></use><use xlink:href="#E2683-MJMAIN-32" x="6492" y="0"></use><use xlink:href="#E2683-MJMAIN-2C" x="6992" y="0"></use><use xlink:href="#E2683-MJMAIN-22EF" x="7436" y="0"></use><use xlink:href="#E2683-MJMAIN-2C" x="8775" y="0"></use><use xlink:href="#E2683-MJMAIN-30" x="9220" y="0"></use></g><g transform="translate(18723,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">以</text></g><g transform="translate(3572,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">下</text></g><g transform="translate(4403,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">操</text></g><g transform="translate(5233,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g><g transform="translate(6064,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">：</text></g></g></g></g><g transform="translate(0,-1640)"><g transform="translate(4000,0)"><use xlink:href="#E2683-MJMAIN-32"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2683-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2683-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2683-MJMAIN-31" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">估</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">回</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">报</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><use xlink:href="#E2683-MJMATHI-55" x="0" y="0"></use><use xlink:href="#E2683-MJMAIN-2190" x="1044" y="0"></use><g transform="translate(2322,0)"><use xlink:href="#E2683-MJMATHI-52" x="0" y="0"></use><g transform="translate(759,-150)"><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-74" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-2B" x="361" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-31" x="1139" y="0"></use></g></g><use xlink:href="#E2683-MJMAIN-2B" x="4562" y="0"></use><use xlink:href="#E2683-MJMATHI-3B3" x="5562" y="0"></use><use xlink:href="#E2683-MJMATHI-55" x="6105" y="0"></use></g><g transform="translate(13912,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-3100)"><g transform="translate(4000,0)"><use xlink:href="#E2683-MJMAIN-32"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2683-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2683-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2683-MJMAIN-32" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">估</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">略</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">梯</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">度</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(8701,0)"><use xlink:href="#E2683-MJMAINB-67" x="0" y="0"></use><g transform="translate(575,412)"><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-29" x="858" y="0"></use></g><use xlink:href="#E2683-MJMAIN-2190" x="1834" y="0"></use><g transform="translate(3112,0)"><use xlink:href="#E2683-MJMAINB-67" x="0" y="0"></use><g transform="translate(575,412)"><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-29" x="858" y="0"></use></g></g><use xlink:href="#E2683-MJMAIN-2B" x="4891" y="0"></use><use xlink:href="#E2683-MJMAIN-5B" x="5891" y="0"></use><use xlink:href="#E2683-MJMATHI-55" x="6169" y="0"></use><use xlink:href="#E2683-MJMAIN-2212" x="7158" y="0"></use><use xlink:href="#E2683-MJMATHI-76" x="8158" y="0"></use><use xlink:href="#E2683-MJMAIN-28" x="8643" y="0"></use><g transform="translate(9032,0)"><use xlink:href="#E2683-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2683-MJMAIN-3B" x="10001" y="0"></use><g transform="translate(10445,0)"><use xlink:href="#E2683-MJMAINB-77" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-2032" x="1175" y="583"></use></g><use xlink:href="#E2683-MJMAIN-29" x="11571" y="0"></use><use xlink:href="#E2683-MJMAIN-5D" x="11960" y="0"></use><use xlink:href="#E2683-MJMAIN-2207" x="12238" y="0"></use><g transform="translate(13238,0)"><use xlink:href="#E2683-MJMAIN-6C"></use><use xlink:href="#E2683-MJMAIN-6E" x="278" y="0"></use></g><use xlink:href="#E2683-MJMATHI-3C0" x="14238" y="0"></use><use xlink:href="#E2683-MJMAIN-28" x="14811" y="0"></use><g transform="translate(15200,0)"><use xlink:href="#E2683-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2683-MJMAIN-2223" x="16583" y="0"></use><g transform="translate(17139,0)"><use xlink:href="#E2683-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2683-MJMAIN-3B" x="18107" y="0"></use><g transform="translate(18552,0)"><use xlink:href="#E2683-MJMATHI-3B8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-2032" x="663" y="583"></use></g><use xlink:href="#E2683-MJMAIN-29" x="19315" y="0"></use></g><g transform="translate(28406,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-4593)"><g transform="translate(4000,0)"><use xlink:href="#E2683-MJMAIN-32"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2683-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2683-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2683-MJMAIN-35" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">估</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">价</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">值</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">梯</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">度</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(8701,0)"><use xlink:href="#E2683-MJMAINB-67" x="0" y="0"></use><g transform="translate(575,412)"><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-29" x="1220" y="0"></use></g><use xlink:href="#E2683-MJMAIN-2190" x="2090" y="0"></use><g transform="translate(3368,0)"><use xlink:href="#E2683-MJMAINB-67" x="0" y="0"></use><g transform="translate(575,412)"><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-29" x="1220" y="0"></use></g></g><use xlink:href="#E2683-MJMAIN-2B" x="5403" y="0"></use><use xlink:href="#E2683-MJMAIN-5B" x="6403" y="0"></use><use xlink:href="#E2683-MJMATHI-55" x="6681" y="0"></use><use xlink:href="#E2683-MJMAIN-2212" x="7670" y="0"></use><use xlink:href="#E2683-MJMATHI-76" x="8670" y="0"></use><use xlink:href="#E2683-MJMAIN-28" x="9155" y="0"></use><g transform="translate(9544,0)"><use xlink:href="#E2683-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2683-MJMAIN-3B" x="10513" y="0"></use><g transform="translate(10957,0)"><use xlink:href="#E2683-MJMAINB-77" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-2032" x="1175" y="583"></use></g><use xlink:href="#E2683-MJMAIN-29" x="12083" y="0"></use><use xlink:href="#E2683-MJMAIN-5D" x="12472" y="0"></use><use xlink:href="#E2683-MJMAIN-2207" x="12750" y="0"></use><use xlink:href="#E2683-MJMATHI-76" x="13583" y="0"></use><use xlink:href="#E2683-MJMAIN-28" x="14068" y="0"></use><g transform="translate(14457,0)"><use xlink:href="#E2683-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2683-MJMAIN-3B" x="15425" y="0"></use><g transform="translate(15870,0)"><use xlink:href="#E2683-MJMAINB-77" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-2032" x="1175" y="583"></use></g><use xlink:href="#E2683-MJMAIN-29" x="16995" y="0"></use></g><g transform="translate(26085,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-5943)"><use xlink:href="#E2683-MJMAIN-33"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><g transform="translate(778,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(1608,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">同</text></g><g transform="translate(2439,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">步</text></g><g transform="translate(3269,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(4100,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4931,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(5761,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(6592,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(7423,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">全</text></g><g transform="translate(8253,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">局</text></g><g transform="translate(9084,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">参</text></g><g transform="translate(9915,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">数</text></g><g transform="translate(10745,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">：</text></g></g><g transform="translate(0,-7386)"><g transform="translate(2000,0)"><use xlink:href="#E2683-MJMAIN-33"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2683-MJMAIN-31" x="778" y="0"></use><g transform="translate(1278,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2108,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(2939,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">略</text></g><g transform="translate(3769,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(4600,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(5431,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(6261,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">用</text></g><g transform="translate(7342,0)"><use xlink:href="#E2683-MJMAINB-67" x="0" y="0"></use><g transform="translate(575,412)"><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-29" x="858" y="0"></use></g></g><g transform="translate(8899,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">全</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">局</text></g><g transform="translate(3572,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(4403,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">略</text></g><g transform="translate(5233,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">参</text></g><g transform="translate(6064,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">数</text></g></g><use xlink:href="#E2683-MJMATHI-3B8" x="16044" y="0"></use><g transform="translate(16513,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">如</text></g></g><g transform="translate(18674,0)"><use xlink:href="#E2683-MJMATHI-3B8" x="0" y="0"></use><use xlink:href="#E2683-MJMAIN-2190" x="746" y="0"></use><use xlink:href="#E2683-MJMATHI-3B8" x="2024" y="0"></use><use xlink:href="#E2683-MJMAIN-2B" x="2715" y="0"></use><g transform="translate(3716,0)"><use xlink:href="#E2683-MJMATHI-3B1" x="0" y="0"></use><g transform="translate(640,412)"><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-29" x="858" y="0"></use></g></g><g transform="translate(5337,0)"><use xlink:href="#E2683-MJMAINB-67" x="0" y="0"></use><g transform="translate(575,412)"><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-29" x="858" y="0"></use></g></g></g><g transform="translate(25569,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-8830)"><g transform="translate(2000,0)"><use xlink:href="#E2683-MJMAIN-33"></use><use xlink:href="#E2683-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2683-MJMAIN-32" x="778" y="0"></use><g transform="translate(1278,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2108,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">价</text></g><g transform="translate(2939,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">值</text></g><g transform="translate(3769,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(4600,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(5431,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(6261,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">用</text></g><g transform="translate(7342,0)"><use xlink:href="#E2683-MJMAINB-67" x="0" y="0"></use><g transform="translate(575,412)"><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-29" x="1220" y="0"></use></g></g><g transform="translate(9155,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">全</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">局</text></g><g transform="translate(3572,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">价</text></g><g transform="translate(4403,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">值</text></g><g transform="translate(5233,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">参</text></g><g transform="translate(6064,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">数</text></g></g><use xlink:href="#E2683-MJMAINB-77" x="16300" y="0"></use><g transform="translate(17131,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">如</text></g></g><g transform="translate(19292,0)"><use xlink:href="#E2683-MJMAINB-77" x="0" y="0"></use><use xlink:href="#E2683-MJMAIN-2190" x="1108" y="0"></use><use xlink:href="#E2683-MJMAINB-77" x="2386" y="0"></use><use xlink:href="#E2683-MJMAIN-2B" x="3439" y="0"></use><g transform="translate(4440,0)"><use xlink:href="#E2683-MJMATHI-3B1" x="0" y="0"></use><g transform="translate(640,412)"><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-29" x="1220" y="0"></use></g></g><g transform="translate(6317,0)"><use xlink:href="#E2683-MJMAINB-67" x="0" y="0"></use><g transform="translate(575,412)"><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2683-MJMAIN-29" x="1220" y="0"></use></g></g></g><g transform="translate(27423,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g></g><g transform="translate(0,-10131)"><g><rect fill="black" stroke="none" width="41597" height="100" x="0" y="-500"></rect></g></g></g></g></g></svg></span></div><script type="math/tex; mode=display" id="MathJax-Element-1227">\; \\ \; \\
\large \textbf{算法 8-3   异步优势执行者/评论者算法（演示某个线程的行为）} \\
\begin{split}
\rule[5pt]{10mm}{0.1em} &\rule[5pt]{265mm}{0.1em} \\
&\cdots \quad \text{同算法 8-1} \quad \cdots \\
&\text{1.（同步全局参数）$\theta' \leftarrow \theta$ ，$\bold w' \leftarrow \bold w$ 。} \\
&\text{2. $\;\,$逐回合执行以下过程：} \\
&\qquad \text{2.1 $\;\,$用策略 $\pi(\theta')$ 生成轨迹 $S_0,A_0,R_1,S_1,\cdots,S_{T-1},A_{T-1},R_T,S_T$ 直到回合结束或达步数上限。} \\
&\qquad \text{2.2 $\;\,$为梯度计算初始化：} \\
&\qquad \qquad \text{2.2.1（初始化目标 $U_T$）若 $S_T$ 是终止状态，则 $U \leftarrow 0$ ，否则 $U \leftarrow v(S_T;\bold w')$ ；} \\
&\qquad \qquad \text{2.2.2（初始化梯度） $\bold g^{(\theta)} \leftarrow 0$ ，$\bold g^{(\bold w)} \leftarrow 0$ 。} \\
&\qquad \text{2.3（异步计算梯度）对 $t=T-1,T-2,\cdots,0$ ，执行以下操作：} \\
&\qquad \qquad \text{2.3.1（估计回报）$U \leftarrow R_{t+1} + \gamma U$ ；} \\
&\qquad \qquad \text{2.3.2（估计策略梯度）$\bold g^{(\theta)} \leftarrow \bold g^{(\theta)} + [U - v(S_t;\bold w')] \nabla \ln \pi(A_t \mid S_t; \theta')$ ；} \\
&\qquad \qquad \text{2.3.5（估计价值梯度）$\bold g^{(\bold w)} \leftarrow \bold g^{(\bold w)} + [U - v(S_t;\bold w')] \nabla v(S_t;\bold w')$ ；} \\
&\text{3.（同步更新）更新全局参数：} \\
&\qquad \text{3.1（策略更新）用 $\bold g^{(\theta)}$ 更新全局策略参数 $\theta$ ，如 $\theta \leftarrow \theta + \alpha^{(\theta)} \bold g^{(\theta)}$ ；} \\
&\qquad \text{3.2（价值更新）用 $\bold g^{(\bold w)}$ 更新全局价值参数 $\bold w$ ，如 $\bold w \leftarrow \bold w + \alpha^{(\bold w)} \bold g^{(\bold w)}$ 。} \\
\rule[-5pt]{10mm}{0.1em} &\rule[-5pt]{265mm}{0.1em}
\end{split}
\; \\ \; \\</script></div></div><p><span>（原作者注：算法 8-3 的步骤 2.3.2 中没有考虑累计折扣是为了遵循论文原文。）</span></p><p><span>执行者 / 评论者算法能够和资格迹结合，与之前函数近似中使用资格迹的方法类似，这里的资格迹也是给对应的函数近似价值参数进行加权更新的。算法 8-4 给出了带资格迹的优势执行者 / 评论者算法。</span></p><div contenteditable="false" spellcheck="false" class="mathjax-block md-end-block md-math-block md-rawblock" id="mathjax-n22" cid="n22" mdtype="math_block"><div class="md-rawblock-container md-math-container" contenteditable="false" tabindex="-1">
						<div class="MathJax_SVG_Display"><span class="MathJax_SVG" id="MathJax-Element-2027-Frame" tabindex="-1" style="font-size: 100%; display: inline-block; zoom: 0.971345;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="101.294ex" height="44.286ex" viewBox="-18.1 -43.5 43612.5 19067.4" role="img" focusable="false" style="vertical-align: -44.185ex; margin-left: -0.042ex; max-width: 100%;"><defs><path stroke-width="0" id="E2784-MJMAINB-38" d="M80 474Q80 561 139 607T278 654Q357 654 411 632Q490 593 494 509Q494 424 416 376L407 371L418 364Q432 356 447 345T481 312T513 260T526 192Q526 100 461 45T285 -11Q184 -11 116 32T48 164Q48 181 50 196T58 225T69 249T84 270T100 286T117 300T134 311T149 321T162 329L152 336Q120 360 100 397T80 474ZM347 404Q404 446 404 503Q404 579 317 599Q309 600 276 600Q178 600 170 538Q170 532 171 527T173 518T178 509T184 501T194 492T205 484T219 476T235 467T254 456T275 445L347 404ZM289 47Q323 47 351 54T402 82T425 137Q425 147 421 161Q411 183 391 197T303 249Q224 293 223 293Q220 291 215 288T197 273T175 248T157 213T149 167Q149 109 188 78T289 47Z"></path><path stroke-width="0" id="E2784-MJMAINB-2D" d="M13 166V278H318V166H13Z"></path><path stroke-width="0" id="E2784-MJMAINB-34" d="M531 0Q510 3 381 3Q238 3 214 0H201V62H313V155H32V217L205 434Q342 606 362 630T387 655L391 656Q395 656 401 656T414 656H427Q447 656 451 645Q453 641 453 429V217H542V155H453V62H542V0H531ZM324 217V494L103 218L213 217H324Z"></path><path stroke-width="0" id="E2784-MJMAINB-2F" d="M451 730Q460 750 479 750Q492 750 501 740T510 718Q508 708 318 244L122 -232Q112 -250 95 -250Q82 -250 73 -241T64 -218Q66 -205 258 261T451 730Z"></path><path stroke-width="0" id="E2784-MJMAIN-22EF" d="M78 250Q78 274 95 292T138 310Q162 310 180 294T199 251Q199 226 182 208T139 190T96 207T78 250ZM525 250Q525 274 542 292T585 310Q609 310 627 294T646 251Q646 226 629 208T586 190T543 207T525 250ZM972 250Q972 274 989 292T1032 310Q1056 310 1074 294T1093 251Q1093 226 1076 208T1033 190T990 207T972 250Z"></path><path stroke-width="0" id="E2784-MJMAIN-38" d="M70 417T70 494T124 618T248 666Q319 666 374 624T429 515Q429 485 418 459T392 417T361 389T335 371T324 363L338 354Q352 344 366 334T382 323Q457 264 457 174Q457 95 399 37T249 -22Q159 -22 101 29T43 155Q43 263 172 335L154 348Q133 361 127 368Q70 417 70 494ZM286 386L292 390Q298 394 301 396T311 403T323 413T334 425T345 438T355 454T364 471T369 491T371 513Q371 556 342 586T275 624Q268 625 242 625Q201 625 165 599T128 534Q128 511 141 492T167 463T217 431Q224 426 228 424L286 386ZM250 21Q308 21 350 55T392 137Q392 154 387 169T375 194T353 216T330 234T301 253T274 270Q260 279 244 289T218 306L210 311Q204 311 181 294T133 239T107 157Q107 98 150 60T250 21Z"></path><path stroke-width="0" id="E2784-MJMAIN-2D" d="M11 179V252H277V179H11Z"></path><path stroke-width="0" id="E2784-MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="0" id="E2784-MJMATHI-3BB" d="M166 673Q166 685 183 694H202Q292 691 316 644Q322 629 373 486T474 207T524 67Q531 47 537 34T546 15T551 6T555 2T556 -2T550 -11H482Q457 3 450 18T399 152L354 277L340 262Q327 246 293 207T236 141Q211 112 174 69Q123 9 111 -1T83 -12Q47 -12 47 20Q47 37 61 52T199 187Q229 216 266 252T321 306L338 322Q338 323 288 462T234 612Q214 657 183 657Q166 657 166 673Z"></path><path stroke-width="0" id="E2784-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2784-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path><path stroke-width="0" id="E2784-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2784-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2784-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2784-MJMAIN-32" d="M109 429Q82 429 66 447T50 491Q50 562 103 614T235 666Q326 666 387 610T449 465Q449 422 429 383T381 315T301 241Q265 210 201 149L142 93L218 92Q375 92 385 97Q392 99 409 186V189H449V186Q448 183 436 95T421 3V0H50V19V31Q50 38 56 46T86 81Q115 113 136 137Q145 147 170 174T204 211T233 244T261 278T284 308T305 340T320 369T333 401T340 431T343 464Q343 527 309 573T212 619Q179 619 154 602T119 569T109 550Q109 549 114 549Q132 549 151 535T170 489Q170 464 154 447T109 429Z"></path><path stroke-width="0" id="E2784-MJMAIN-2E" d="M78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60Z"></path><path stroke-width="0" id="E2784-MJMAINB-7A" d="M48 262Q48 264 54 349T60 436V444H252Q289 444 336 444T394 445Q441 445 450 441T459 418Q459 406 458 404Q456 399 327 229T194 55H237Q260 56 268 56T297 58T325 65T348 77T370 98T384 128T395 170Q400 197 400 216Q400 217 431 217H462V211Q461 208 453 108T444 6V0H245Q46 0 43 2Q32 7 32 28V33Q32 41 40 52T84 112Q129 170 164 217L298 393H256Q189 392 165 380Q124 360 115 303Q110 280 110 256Q110 254 79 254H48V262Z"></path><path stroke-width="0" id="E2784-MJMAIN-2190" d="M944 261T944 250T929 230H165Q167 228 182 216T211 189T244 152T277 96T303 25Q308 7 308 0Q308 -11 288 -11Q281 -11 278 -11T272 -7T267 2T263 21Q245 94 195 151T73 236Q58 242 55 247Q55 254 59 257T73 264Q121 283 158 314T215 375T247 434T264 480L267 497Q269 503 270 505T275 509T288 511Q308 511 308 500Q308 493 303 475Q293 438 278 406T246 352T215 315T185 287T165 270H929Q944 261 944 250Z"></path><path stroke-width="0" id="E2784-MJMAIN-30" d="M96 585Q152 666 249 666Q297 666 345 640T423 548Q460 465 460 320Q460 165 417 83Q397 41 362 16T301 -15T250 -22Q224 -22 198 -16T137 16T82 83Q39 165 39 320Q39 494 96 585ZM321 597Q291 629 250 629Q208 629 178 597Q153 571 145 525T137 333Q137 175 145 125T181 46Q209 16 250 16Q290 16 318 46Q347 76 354 130T362 333Q362 478 354 524T321 597Z"></path><path stroke-width="0" id="E2784-MJMATHI-49" d="M43 1Q26 1 26 10Q26 12 29 24Q34 43 39 45Q42 46 54 46H60Q120 46 136 53Q137 53 138 54Q143 56 149 77T198 273Q210 318 216 344Q286 624 286 626Q284 630 284 631Q274 637 213 637H193Q184 643 189 662Q193 677 195 680T209 683H213Q285 681 359 681Q481 681 487 683H497Q504 676 504 672T501 655T494 639Q491 637 471 637Q440 637 407 634Q393 631 388 623Q381 609 337 432Q326 385 315 341Q245 65 245 59Q245 52 255 50T307 46H339Q345 38 345 37T342 19Q338 6 332 0H316Q279 2 179 2Q143 2 113 2T65 2T43 1Z"></path><path stroke-width="0" id="E2784-MJMAIN-33" d="M127 463Q100 463 85 480T69 524Q69 579 117 622T233 665Q268 665 277 664Q351 652 390 611T430 522Q430 470 396 421T302 350L299 348Q299 347 308 345T337 336T375 315Q457 262 457 175Q457 96 395 37T238 -22Q158 -22 100 21T42 130Q42 158 60 175T105 193Q133 193 151 175T169 130Q169 119 166 110T159 94T148 82T136 74T126 70T118 67L114 66Q165 21 238 21Q293 21 321 74Q338 107 338 175V195Q338 290 274 322Q259 328 213 329L171 330L168 332Q166 335 166 348Q166 366 174 366Q202 366 232 371Q266 376 294 413T322 525V533Q322 590 287 612Q265 626 240 626Q208 626 181 615T143 592T132 580H135Q138 579 143 578T153 573T165 566T175 555T183 540T186 520Q186 498 172 481T127 463Z"></path><path stroke-width="0" id="E2784-MJMAIN-34" d="M462 0Q444 3 333 3Q217 3 199 0H190V46H221Q241 46 248 46T265 48T279 53T286 61Q287 63 287 115V165H28V211L179 442Q332 674 334 675Q336 677 355 677H373L379 671V211H471V165H379V114Q379 73 379 66T385 54Q393 47 442 46H471V0H462ZM293 211V545L74 212L183 211H293Z"></path><path stroke-width="0" id="E2784-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2784-MJMAIN-2B" d="M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"></path><path stroke-width="0" id="E2784-MJMAIN-2207" d="M46 676Q46 679 51 683H781Q786 679 786 676Q786 674 617 326T444 -26Q439 -33 416 -33T388 -26Q385 -22 216 326T46 676ZM697 596Q697 597 445 597T193 596Q195 591 319 336T445 80L697 596Z"></path><path stroke-width="0" id="E2784-MJMAIN-6C" d="M42 46H56Q95 46 103 60V68Q103 77 103 91T103 124T104 167T104 217T104 272T104 329Q104 366 104 407T104 482T104 542T103 586T103 603Q100 622 89 628T44 637H26V660Q26 683 28 683L38 684Q48 685 67 686T104 688Q121 689 141 690T171 693T182 694H185V379Q185 62 186 60Q190 52 198 49Q219 46 247 46H263V0H255L232 1Q209 2 183 2T145 3T107 3T57 1L34 0H26V46H42Z"></path><path stroke-width="0" id="E2784-MJMAIN-6E" d="M41 46H55Q94 46 102 60V68Q102 77 102 91T102 122T103 161T103 203Q103 234 103 269T102 328V351Q99 370 88 376T43 385H25V408Q25 431 27 431L37 432Q47 433 65 434T102 436Q119 437 138 438T167 441T178 442H181V402Q181 364 182 364T187 369T199 384T218 402T247 421T285 437Q305 442 336 442Q450 438 463 329Q464 322 464 190V104Q464 66 466 59T477 49Q498 46 526 46H542V0H534L510 1Q487 2 460 2T422 3Q319 3 310 0H302V46H318Q379 46 379 62Q380 64 380 200Q379 335 378 343Q372 371 358 385T334 402T308 404Q263 404 229 370Q202 343 195 315T187 232V168V108Q187 78 188 68T191 55T200 49Q221 46 249 46H265V0H257L234 1Q210 2 183 2T145 3Q42 3 33 0H25V46H41Z"></path><path stroke-width="0" id="E2784-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2784-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2784-MJMAIN-2223" d="M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z"></path><path stroke-width="0" id="E2784-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2784-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2784-MJMAIN-35" d="M164 157Q164 133 148 117T109 101H102Q148 22 224 22Q294 22 326 82Q345 115 345 210Q345 313 318 349Q292 382 260 382H254Q176 382 136 314Q132 307 129 306T114 304Q97 304 95 310Q93 314 93 485V614Q93 664 98 664Q100 666 102 666Q103 666 123 658T178 642T253 634Q324 634 389 662Q397 666 402 666Q410 666 410 648V635Q328 538 205 538Q174 538 149 544L139 546V374Q158 388 169 396T205 412T256 420Q337 420 393 355T449 201Q449 109 385 44T229 -22Q148 -22 99 32T50 154Q50 178 61 192T84 210T107 214Q132 214 148 197T164 157Z"></path><path stroke-width="0" id="E2784-MJMATHI-3B1" d="M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z"></path><path stroke-width="0" id="E2784-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2784-MJMATHI-55" d="M107 637Q73 637 71 641Q70 643 70 649Q70 673 81 682Q83 683 98 683Q139 681 234 681Q268 681 297 681T342 682T362 682Q378 682 378 672Q378 670 376 658Q371 641 366 638H364Q362 638 359 638T352 638T343 637T334 637Q295 636 284 634T266 623Q265 621 238 518T184 302T154 169Q152 155 152 140Q152 86 183 55T269 24Q336 24 403 69T501 205L552 406Q599 598 599 606Q599 633 535 637Q511 637 511 648Q511 650 513 660Q517 676 519 679T529 683Q532 683 561 682T645 680Q696 680 723 681T752 682Q767 682 767 672Q767 650 759 642Q756 637 737 637Q666 633 648 597Q646 592 598 404Q557 235 548 205Q515 105 433 42T263 -22Q171 -22 116 34T60 167V183Q60 201 115 421Q164 622 164 628Q164 635 107 637Z"></path><path stroke-width="0" id="E2784-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2784-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2784-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path><path stroke-width="0" id="E2784-MJMAIN-36" d="M42 313Q42 476 123 571T303 666Q372 666 402 630T432 550Q432 525 418 510T379 495Q356 495 341 509T326 548Q326 592 373 601Q351 623 311 626Q240 626 194 566Q147 500 147 364L148 360Q153 366 156 373Q197 433 263 433H267Q313 433 348 414Q372 400 396 374T435 317Q456 268 456 210V192Q456 169 451 149Q440 90 387 34T253 -22Q225 -22 199 -14T143 16T92 75T56 172T42 313ZM257 397Q227 397 205 380T171 335T154 278T148 216Q148 133 160 97T198 39Q222 21 251 21Q302 21 329 59Q342 77 347 104T352 209Q352 289 347 316T329 361Q302 397 257 397Z"></path><path stroke-width="0" id="E2784-MJMAIN-37" d="M55 458Q56 460 72 567L88 674Q88 676 108 676H128V672Q128 662 143 655T195 646T364 644H485V605L417 512Q408 500 387 472T360 435T339 403T319 367T305 330T292 284T284 230T278 162T275 80Q275 66 275 52T274 28V19Q270 2 255 -10T221 -22Q210 -22 200 -19T179 0T168 40Q168 198 265 368Q285 400 349 489L395 552H302Q128 552 119 546Q113 543 108 522T98 479L95 458V455H55V458Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><g transform="translate(10501,-2531)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">算</text><g transform="translate(1052,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">法</text></g><use transform="scale(1.2)" xlink:href="#E2784-MJMAINB-38" x="1963" y="0"></use><use transform="scale(1.2)" xlink:href="#E2784-MJMAINB-2D" x="2538" y="0"></use><use transform="scale(1.2)" xlink:href="#E2784-MJMAINB-34" x="2921" y="0"></use><g transform="translate(4945,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">带</text></g><g transform="translate(5998,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">资</text></g><g transform="translate(7014,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">格</text></g><g transform="translate(8067,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">迹</text></g><g transform="translate(9120,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">的</text></g><g transform="translate(10136,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">优</text></g><g transform="translate(11189,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">势</text></g><g transform="translate(12205,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(13258,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(14311,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">者</text></g><use transform="scale(1.2)" xlink:href="#E2784-MJMAINB-2F" x="12803" y="0"></use><g transform="translate(16053,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">评</text></g><g transform="translate(17106,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">论</text></g><g transform="translate(18159,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">者</text></g><g transform="translate(19212,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(20265,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">法</text></g></g><g transform="translate(0,-10541)"><g transform="translate(-19,0)"><g transform="translate(0,6479)"><g><rect fill="black" stroke="none" width="1569" height="100" x="0" y="500"></rect></g></g><g transform="translate(0,-6280)"><g><rect fill="black" stroke="none" width="1569" height="100" x="0" y="-500"></rect></g></g></g><g transform="translate(1551,0)"><g transform="translate(0,6479)"><g><rect fill="black" stroke="none" width="41597" height="100" x="0" y="500"></rect></g></g><g transform="translate(0,5036)"><use xlink:href="#E2784-MJMAIN-22EF" x="166" y="0"></use><g transform="translate(2505,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">同</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">法</text></g><use xlink:href="#E2784-MJMAIN-38" x="2741" y="0"></use><use xlink:href="#E2784-MJMAIN-2D" x="3241" y="0"></use><use xlink:href="#E2784-MJMAIN-31" x="3574" y="0"></use><g transform="translate(4074,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(4905,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">但</text></g><g transform="translate(5736,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">加</text></g><g transform="translate(6566,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">两</text></g><g transform="translate(7397,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">个</text></g><g transform="translate(8228,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">资</text></g><g transform="translate(9058,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">格</text></g><g transform="translate(9889,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">迹</text></g><g transform="translate(10720,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">参</text></g><g transform="translate(11550,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">数</text></g><g transform="translate(12381,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">：</text></g><g transform="translate(13462,0)"><use xlink:href="#E2784-MJMATHI-3BB" x="0" y="0"></use><g transform="translate(583,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="858" y="0"></use></g><use xlink:href="#E2784-MJMAIN-2C" x="1564" y="0"></use><g transform="translate(2009,0)"><use xlink:href="#E2784-MJMATHI-3BB" x="0" y="0"></use><g transform="translate(583,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="1220" y="0"></use></g></g></g><g transform="translate(17292,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g><use xlink:href="#E2784-MJMAIN-22EF" x="22044" y="0"></use></g><g transform="translate(0,3593)"><use xlink:href="#E2784-MJMAIN-32"></use><use xlink:href="#E2784-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2784-MJMAIN-31" x="778" y="0"></use><g transform="translate(1278,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2108,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">初</text></g><g transform="translate(2939,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">始</text></g><g transform="translate(3769,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">化</text></g><g transform="translate(4600,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">资</text></g><g transform="translate(5431,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">格</text></g><g transform="translate(6261,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">迹</text></g><g transform="translate(7092,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">和</text></g><g transform="translate(7923,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">累</text></g><g transform="translate(8753,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(9584,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">折</text></g><g transform="translate(10415,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">扣</text></g><g transform="translate(11245,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(12076,0)"><use xlink:href="#E2784-MJMAINB-7A" x="0" y="0"></use><g transform="translate(511,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="858" y="0"></use></g><use xlink:href="#E2784-MJMAIN-2190" x="1770" y="0"></use><use xlink:href="#E2784-MJMAIN-30" x="3048" y="0"></use></g><g transform="translate(15624,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g></g><g transform="translate(16705,0)"><use xlink:href="#E2784-MJMAINB-7A" x="0" y="0"></use><g transform="translate(511,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="1220" y="0"></use></g><use xlink:href="#E2784-MJMAIN-2190" x="2026" y="0"></use><use xlink:href="#E2784-MJMAIN-30" x="3304" y="0"></use></g><g transform="translate(20509,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g></g><g transform="translate(21590,0)"><use xlink:href="#E2784-MJMATHI-49" x="0" y="0"></use><use xlink:href="#E2784-MJMAIN-2190" x="781" y="0"></use><use xlink:href="#E2784-MJMAIN-31" x="2059" y="0"></use></g><g transform="translate(24149,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g><g transform="translate(0,2293)"><use xlink:href="#E2784-MJMAIN-22EF" x="166" y="0"></use><g transform="translate(2505,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">同</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">法</text></g><use xlink:href="#E2784-MJMAIN-38" x="2741" y="0"></use><use xlink:href="#E2784-MJMAIN-2D" x="3241" y="0"></use><use xlink:href="#E2784-MJMAIN-32" x="3574" y="0"></use></g><use xlink:href="#E2784-MJMAIN-22EF" x="7746" y="0"></use></g><g transform="translate(0,850)"><g transform="translate(2000,0)"><use xlink:href="#E2784-MJMAIN-32"></use><use xlink:href="#E2784-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2784-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2784-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2784-MJMAIN-34" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">略</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">资</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">格</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">迹</text></g><g transform="translate(8701,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(9531,0)"><use xlink:href="#E2784-MJMAINB-7A" x="0" y="0"></use><g transform="translate(511,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="858" y="0"></use></g><use xlink:href="#E2784-MJMAIN-2190" x="1770" y="0"></use><use xlink:href="#E2784-MJMATHI-3B3" x="3048" y="0"></use><g transform="translate(3591,0)"><use xlink:href="#E2784-MJMATHI-3BB" x="0" y="0"></use><g transform="translate(583,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="858" y="0"></use></g></g><g transform="translate(5156,0)"><use xlink:href="#E2784-MJMAINB-7A" x="0" y="0"></use><g transform="translate(511,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="858" y="0"></use></g></g><use xlink:href="#E2784-MJMAIN-2B" x="6871" y="0"></use><use xlink:href="#E2784-MJMATHI-49" x="7871" y="0"></use><use xlink:href="#E2784-MJMAIN-2207" x="8375" y="0"></use><g transform="translate(9374,0)"><use xlink:href="#E2784-MJMAIN-6C"></use><use xlink:href="#E2784-MJMAIN-6E" x="278" y="0"></use></g><use xlink:href="#E2784-MJMATHI-3C0" x="10375" y="0"></use><use xlink:href="#E2784-MJMAIN-28" x="10948" y="0"></use><use xlink:href="#E2784-MJMATHI-41" x="11337" y="0"></use><use xlink:href="#E2784-MJMAIN-2223" x="12365" y="0"></use><use xlink:href="#E2784-MJMATHI-53" x="12921" y="0"></use><use xlink:href="#E2784-MJMAIN-3B" x="13566" y="0"></use><use xlink:href="#E2784-MJMAINB-77" x="14010" y="0"></use><use xlink:href="#E2784-MJMAIN-29" x="14841" y="0"></use></g><g transform="translate(24762,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-644)"><g transform="translate(2000,0)"><use xlink:href="#E2784-MJMAIN-32"></use><use xlink:href="#E2784-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2784-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2784-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2784-MJMAIN-35" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">略</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">改</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">进</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><use xlink:href="#E2784-MJMATHI-3B8" x="0" y="0"></use><use xlink:href="#E2784-MJMAIN-2190" x="746" y="0"></use><use xlink:href="#E2784-MJMATHI-3B8" x="2024" y="0"></use><use xlink:href="#E2784-MJMAIN-2B" x="2715" y="0"></use><g transform="translate(3716,0)"><use xlink:href="#E2784-MJMATHI-3B1" x="0" y="0"></use><g transform="translate(640,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="858" y="0"></use></g></g><use xlink:href="#E2784-MJMAIN-5B" x="5337" y="0"></use><use xlink:href="#E2784-MJMATHI-55" x="5615" y="0"></use><use xlink:href="#E2784-MJMAIN-2212" x="6604" y="0"></use><use xlink:href="#E2784-MJMATHI-76" x="7605" y="0"></use><use xlink:href="#E2784-MJMAIN-28" x="8090" y="0"></use><use xlink:href="#E2784-MJMATHI-53" x="8479" y="0"></use><use xlink:href="#E2784-MJMAIN-3B" x="9124" y="0"></use><use xlink:href="#E2784-MJMAINB-77" x="9568" y="0"></use><use xlink:href="#E2784-MJMAIN-29" x="10399" y="0"></use><use xlink:href="#E2784-MJMAIN-5D" x="10788" y="0"></use><g transform="translate(11066,0)"><use xlink:href="#E2784-MJMAINB-7A" x="0" y="0"></use><g transform="translate(511,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="858" y="0"></use></g></g></g><g transform="translate(19599,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-2137)"><g transform="translate(2000,0)"><use xlink:href="#E2784-MJMAIN-32"></use><use xlink:href="#E2784-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2784-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2784-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2784-MJMAIN-36" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">价</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">值</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">资</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">格</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">迹</text></g><g transform="translate(8701,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(9531,0)"><use xlink:href="#E2784-MJMAINB-7A" x="0" y="0"></use><g transform="translate(511,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="1220" y="0"></use></g><use xlink:href="#E2784-MJMAIN-2190" x="2026" y="0"></use><use xlink:href="#E2784-MJMATHI-3B3" x="3304" y="0"></use><g transform="translate(3847,0)"><use xlink:href="#E2784-MJMATHI-3BB" x="0" y="0"></use><g transform="translate(583,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="1220" y="0"></use></g></g><g transform="translate(5668,0)"><use xlink:href="#E2784-MJMAINB-7A" x="0" y="0"></use><g transform="translate(511,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="1220" y="0"></use></g></g><use xlink:href="#E2784-MJMAIN-2B" x="7638" y="0"></use><use xlink:href="#E2784-MJMAIN-2207" x="8639" y="0"></use><use xlink:href="#E2784-MJMATHI-76" x="9472" y="0"></use><use xlink:href="#E2784-MJMAIN-28" x="9957" y="0"></use><use xlink:href="#E2784-MJMATHI-53" x="10346" y="0"></use><use xlink:href="#E2784-MJMAIN-3B" x="10991" y="0"></use><use xlink:href="#E2784-MJMAINB-77" x="11435" y="0"></use><use xlink:href="#E2784-MJMAIN-29" x="12266" y="0"></use></g><g transform="translate(22187,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-3630)"><g transform="translate(2000,0)"><use xlink:href="#E2784-MJMAIN-32"></use><use xlink:href="#E2784-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2784-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2784-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2784-MJMAIN-37" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">价</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">值</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><use xlink:href="#E2784-MJMAINB-77" x="0" y="0"></use><use xlink:href="#E2784-MJMAIN-2190" x="1108" y="0"></use><use xlink:href="#E2784-MJMAINB-77" x="2386" y="0"></use><use xlink:href="#E2784-MJMAIN-2B" x="3439" y="0"></use><g transform="translate(4440,0)"><use xlink:href="#E2784-MJMATHI-3B1" x="0" y="0"></use><g transform="translate(640,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="1220" y="0"></use></g></g><use xlink:href="#E2784-MJMAIN-5B" x="6317" y="0"></use><use xlink:href="#E2784-MJMATHI-55" x="6595" y="0"></use><use xlink:href="#E2784-MJMAIN-2212" x="7584" y="0"></use><use xlink:href="#E2784-MJMATHI-76" x="8585" y="0"></use><use xlink:href="#E2784-MJMAIN-28" x="9070" y="0"></use><use xlink:href="#E2784-MJMATHI-53" x="9459" y="0"></use><use xlink:href="#E2784-MJMAIN-3B" x="10104" y="0"></use><use xlink:href="#E2784-MJMAINB-77" x="10548" y="0"></use><use xlink:href="#E2784-MJMAIN-29" x="11379" y="0"></use><use xlink:href="#E2784-MJMAIN-5D" x="11768" y="0"></use><g transform="translate(12046,0)"><use xlink:href="#E2784-MJMAINB-7A" x="0" y="0"></use><g transform="translate(511,412)"><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2784-MJMAIN-29" x="1220" y="0"></use></g></g></g><g transform="translate(20835,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-4980)"><use xlink:href="#E2784-MJMAIN-22EF" x="166" y="0"></use><g transform="translate(2505,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">同</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">法</text></g><use xlink:href="#E2784-MJMAIN-38" x="2741" y="0"></use><use xlink:href="#E2784-MJMAIN-2D" x="3241" y="0"></use><use xlink:href="#E2784-MJMAIN-32" x="3574" y="0"></use></g><use xlink:href="#E2784-MJMAIN-22EF" x="7746" y="0"></use></g><g transform="translate(0,-6280)"><g><rect fill="black" stroke="none" width="41597" height="100" x="0" y="-500"></rect></g></g></g></g></g></svg></span></div><script type="math/tex; mode=display" id="MathJax-Element-2027">\; \\ \; \\
\large \textbf{算法 8-4   带资格迹的优势执行者/评论者算法} \\
\begin{split}
\rule[5pt]{10mm}{0.1em} &\rule[5pt]{265mm}{0.1em} \\
&\cdots \quad \text{同算法 8-1，但加两个资格迹参数： $\lambda^{(\theta)}, \lambda^{(\bold w)}$ 。} \quad \cdots \\
&\text{2.1（初始化资格迹和累计折扣）$\bold z^{(\theta)} \leftarrow 0$ ，$\bold z^{(\bold w)} \leftarrow 0$ ，$I \leftarrow 1$ 。} \\
&\cdots \quad \text{同算法 8-2} \quad \cdots \\
&\qquad \text{2.3.4（更新策略资格迹）$\bold z^{(\theta)} \leftarrow \gamma \lambda^{(\theta)} \bold z^{(\theta)} + I\nabla \ln\pi(A \mid S; \bold w)$ ；} \\
&\qquad \text{2.3.5（策略改进）$\theta \leftarrow \theta + \alpha^{(\theta)} [U-v(S;\bold w)] \bold z^{(\theta)}$ ；} \\
&\qquad \text{2.3.6（更新价值资格迹）$\bold z^{(\bold w)} \leftarrow \gamma \lambda^{(\bold w)} \bold z^{(\bold w)} + \nabla v(S; \bold w)$ ；} \\
&\qquad \text{2.3.7（更新价值）$\bold w \leftarrow \bold w + \alpha^{(\bold w)} [U - v(S;\bold w)] \bold z^{(\bold w)}$ ；} \\
&\cdots \quad \text{同算法 8-2} \quad \cdots \\
\rule[-5pt]{10mm}{0.1em} &\rule[-5pt]{265mm}{0.1em}
\end{split}
\; \\ \; \\</script>
					</div></div><h3><a name="二基于代理优势的同策算法" class="md-header-anchor"></a><span>二、基于代理优势的同策算法</span></h3><p><span>在迭代过程中不直接优化期望目标，而是试图优化期望目标的近似——代理优势，这类算法称为面向代理优势的执行者 / 评论者算法。在很多问题上，这些算法会比简单的执行者 / 评论者算法得到更好的性能。</span></p><p><span>暂略</span></p><h3><a name="三信任域算法" class="md-header-anchor"></a><span>三、信任域算法</span></h3><p><span>暂略</span></p><h3><a name="四重要性采样异策执行者--评论者算法" class="md-header-anchor"></a><span>四、重要性采样异策执行者 / 评论者算法</span></h3><p><span>执行者 / 评论者算法可以和重要性采样结合，得到</span><strong><span>异策的执行者 / 评论家算法</span></strong><span>（Off-Policy Actor-Critic, OffPAC）。用 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="6.031ex" height="2.71ex" viewBox="0 -832.7 2596.6 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2712-MJMATHI-62" d="M73 647Q73 657 77 670T89 683Q90 683 161 688T234 694Q246 694 246 685T212 542Q204 508 195 472T180 418L176 399Q176 396 182 402Q231 442 283 442Q345 442 383 396T422 280Q422 169 343 79T173 -11Q123 -11 82 27T40 150V159Q40 180 48 217T97 414Q147 611 147 623T109 637Q104 637 101 637H96Q86 637 83 637T76 640T73 647ZM336 325V331Q336 405 275 405Q258 405 240 397T207 376T181 352T163 330L157 322L136 236Q114 150 114 114Q114 66 138 42Q154 26 178 26Q211 26 245 58Q270 81 285 114T318 219Q336 291 336 325Z"></path><path stroke-width="0" id="E2712-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2712-MJMAIN-22C5" d="M78 250Q78 274 95 292T138 310Q162 310 180 294T199 251Q199 226 182 208T139 190T96 207T78 250Z"></path><path stroke-width="0" id="E2712-MJMAIN-2223" d="M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z"></path><path stroke-width="0" id="E2712-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2712-MJMATHI-62" x="0" y="0"></use><use xlink:href="#E2712-MJMAIN-28" x="429" y="0"></use><use xlink:href="#E2712-MJMAIN-22C5" x="818" y="0"></use><use xlink:href="#E2712-MJMAIN-2223" x="1373" y="0"></use><use xlink:href="#E2712-MJMAIN-22C5" x="1929" y="0"></use><use xlink:href="#E2712-MJMAIN-29" x="2207" y="0"></use></g></svg></span><script type="math/tex">b(\cdot \mid \cdot)</script><span> 表示行为策略，则梯度方向可由 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="25.384ex" height="2.903ex" viewBox="0 -832.7 10929.3 1250" role="img" focusable="false" style="vertical-align: -0.969ex;"><defs><path stroke-width="0" id="E2713-MJMAIN-45" d="M128 619Q121 626 117 628T101 631T58 634H25V680H597V676Q599 670 611 560T625 444V440H585V444Q584 447 582 465Q578 500 570 526T553 571T528 601T498 619T457 629T411 633T353 634Q266 634 251 633T233 622Q233 622 233 621Q232 619 232 497V376H286Q359 378 377 385Q413 401 416 469Q416 471 416 473V493H456V213H416V233Q415 268 408 288T383 317T349 328T297 330Q290 330 286 330H232V196V114Q232 57 237 52Q243 47 289 47H340H391Q428 47 452 50T505 62T552 92T584 146Q594 172 599 200T607 247T612 270V273H652V270Q651 267 632 137T610 3V0H25V46H58Q100 47 109 49T128 61V619Z"></path><path stroke-width="0" id="E2713-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2713-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2713-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path><path stroke-width="0" id="E2713-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2713-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2713-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2713-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2713-MJMAIN-2207" d="M46 676Q46 679 51 683H781Q786 679 786 676Q786 674 617 326T444 -26Q439 -33 416 -33T388 -26Q385 -22 216 326T46 676ZM697 596Q697 597 445 597T193 596Q195 591 319 336T445 80L697 596Z"></path><path stroke-width="0" id="E2713-MJMAIN-6C" d="M42 46H56Q95 46 103 60V68Q103 77 103 91T103 124T104 167T104 217T104 272T104 329Q104 366 104 407T104 482T104 542T103 586T103 603Q100 622 89 628T44 637H26V660Q26 683 28 683L38 684Q48 685 67 686T104 688Q121 689 141 690T171 693T182 694H185V379Q185 62 186 60Q190 52 198 49Q219 46 247 46H263V0H255L232 1Q209 2 183 2T145 3T107 3T57 1L34 0H26V46H42Z"></path><path stroke-width="0" id="E2713-MJMAIN-6E" d="M41 46H55Q94 46 102 60V68Q102 77 102 91T102 122T103 161T103 203Q103 234 103 269T102 328V351Q99 370 88 376T43 385H25V408Q25 431 27 431L37 432Q47 433 65 434T102 436Q119 437 138 438T167 441T178 442H181V402Q181 364 182 364T187 369T199 384T218 402T247 421T285 437Q305 442 336 442Q450 438 463 329Q464 322 464 190V104Q464 66 466 59T477 49Q498 46 526 46H542V0H534L510 1Q487 2 460 2T422 3Q319 3 310 0H302V46H318Q379 46 379 62Q380 64 380 200Q379 335 378 343Q372 371 358 385T334 402T308 404Q263 404 229 370Q202 343 195 315T187 232V168V108Q187 78 188 68T191 55T200 49Q221 46 249 46H265V0H257L234 1Q210 2 183 2T145 3Q42 3 33 0H25V46H41Z"></path><path stroke-width="0" id="E2713-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2713-MJMAIN-2223" d="M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z"></path><path stroke-width="0" id="E2713-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2713-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2713-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2713-MJMAIN-45" x="0" y="0"></use><g transform="translate(681,-186)"><use transform="scale(0.707)" xlink:href="#E2713-MJMATHI-3C0" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2713-MJMAIN-28" x="573" y="0"></use><use transform="scale(0.707)" xlink:href="#E2713-MJMATHI-3B8" x="962" y="0"></use><use transform="scale(0.707)" xlink:href="#E2713-MJMAIN-29" x="1431" y="0"></use></g><use xlink:href="#E2713-MJMAIN-5B" x="2067" y="0"></use><g transform="translate(2345,0)"><use xlink:href="#E2713-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2713-MJMATHI-74" x="1100" y="-213"></use></g><use xlink:href="#E2713-MJMAIN-2207" x="3479" y="0"></use><g transform="translate(4478,0)"><use xlink:href="#E2713-MJMAIN-6C"></use><use xlink:href="#E2713-MJMAIN-6E" x="278" y="0"></use></g><use xlink:href="#E2713-MJMATHI-3C0" x="5479" y="0"></use><use xlink:href="#E2713-MJMAIN-28" x="6052" y="0"></use><g transform="translate(6441,0)"><use xlink:href="#E2713-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2713-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2713-MJMAIN-2223" x="7824" y="0"></use><g transform="translate(8380,0)"><use xlink:href="#E2713-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2713-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2713-MJMAIN-3B" x="9348" y="0"></use><use xlink:href="#E2713-MJMATHI-3B8" x="9793" y="0"></use><use xlink:href="#E2713-MJMAIN-29" x="10262" y="0"></use><use xlink:href="#E2713-MJMAIN-5D" x="10651" y="0"></use></g></svg></span><script type="math/tex">{\rm E}_{\pi(\theta)}[\Psi_t \nabla\ln\pi(A_t \mid S_t;\theta)]</script><span> 变为 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="37.496ex" height="5.797ex" viewBox="0 -1497.2 16144.1 2496" role="img" focusable="false" style="vertical-align: -2.32ex;"><defs><path stroke-width="0" id="E2714-MJMAIN-45" d="M128 619Q121 626 117 628T101 631T58 634H25V680H597V676Q599 670 611 560T625 444V440H585V444Q584 447 582 465Q578 500 570 526T553 571T528 601T498 619T457 629T411 633T353 634Q266 634 251 633T233 622Q233 622 233 621Q232 619 232 497V376H286Q359 378 377 385Q413 401 416 469Q416 471 416 473V493H456V213H416V233Q415 268 408 288T383 317T349 328T297 330Q290 330 286 330H232V196V114Q232 57 237 52Q243 47 289 47H340H391Q428 47 452 50T505 62T552 92T584 146Q594 172 599 200T607 247T612 270V273H652V270Q651 267 632 137T610 3V0H25V46H58Q100 47 109 49T128 61V619Z"></path><path stroke-width="0" id="E2714-MJMATHI-62" d="M73 647Q73 657 77 670T89 683Q90 683 161 688T234 694Q246 694 246 685T212 542Q204 508 195 472T180 418L176 399Q176 396 182 402Q231 442 283 442Q345 442 383 396T422 280Q422 169 343 79T173 -11Q123 -11 82 27T40 150V159Q40 180 48 217T97 414Q147 611 147 623T109 637Q104 637 101 637H96Q86 637 83 637T76 640T73 647ZM336 325V331Q336 405 275 405Q258 405 240 397T207 376T181 352T163 330L157 322L136 236Q114 150 114 114Q114 66 138 42Q154 26 178 26Q211 26 245 58Q270 81 285 114T318 219Q336 291 336 325Z"></path><path stroke-width="0" id="E2714-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2714-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2714-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2714-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2714-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2714-MJMAIN-2223" d="M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z"></path><path stroke-width="0" id="E2714-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2714-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2714-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path><path stroke-width="0" id="E2714-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2714-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2714-MJMAIN-2207" d="M46 676Q46 679 51 683H781Q786 679 786 676Q786 674 617 326T444 -26Q439 -33 416 -33T388 -26Q385 -22 216 326T46 676ZM697 596Q697 597 445 597T193 596Q195 591 319 336T445 80L697 596Z"></path><path stroke-width="0" id="E2714-MJMAIN-6C" d="M42 46H56Q95 46 103 60V68Q103 77 103 91T103 124T104 167T104 217T104 272T104 329Q104 366 104 407T104 482T104 542T103 586T103 603Q100 622 89 628T44 637H26V660Q26 683 28 683L38 684Q48 685 67 686T104 688Q121 689 141 690T171 693T182 694H185V379Q185 62 186 60Q190 52 198 49Q219 46 247 46H263V0H255L232 1Q209 2 183 2T145 3T107 3T57 1L34 0H26V46H42Z"></path><path stroke-width="0" id="E2714-MJMAIN-6E" d="M41 46H55Q94 46 102 60V68Q102 77 102 91T102 122T103 161T103 203Q103 234 103 269T102 328V351Q99 370 88 376T43 385H25V408Q25 431 27 431L37 432Q47 433 65 434T102 436Q119 437 138 438T167 441T178 442H181V402Q181 364 182 364T187 369T199 384T218 402T247 421T285 437Q305 442 336 442Q450 438 463 329Q464 322 464 190V104Q464 66 466 59T477 49Q498 46 526 46H542V0H534L510 1Q487 2 460 2T422 3Q319 3 310 0H302V46H318Q379 46 379 62Q380 64 380 200Q379 335 378 343Q372 371 358 385T334 402T308 404Q263 404 229 370Q202 343 195 315T187 232V168V108Q187 78 188 68T191 55T200 49Q221 46 249 46H265V0H257L234 1Q210 2 183 2T145 3Q42 3 33 0H25V46H41Z"></path><path stroke-width="0" id="E2714-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path><path stroke-width="0" id="E2714-MJSZ3-5B" d="M247 -949V1450H516V1388H309V-887H516V-949H247Z"></path><path stroke-width="0" id="E2714-MJSZ3-5D" d="M11 1388V1450H280V-949H11V-887H218V1388H11Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2714-MJMAIN-45" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2714-MJMATHI-62" x="963" y="-213"></use><g transform="translate(1251,0)"><use xlink:href="#E2714-MJSZ3-5B"></use><g transform="translate(528,0)"><g transform="translate(120,0)"><rect stroke="none" width="5291" height="60" x="0" y="220"></rect><g transform="translate(60,693)"><use xlink:href="#E2714-MJMATHI-3C0" x="0" y="0"></use><use xlink:href="#E2714-MJMAIN-28" x="573" y="0"></use><g transform="translate(962,0)"><use xlink:href="#E2714-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2714-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2714-MJMAIN-2223" x="2345" y="0"></use><g transform="translate(2900,0)"><use xlink:href="#E2714-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2714-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2714-MJMAIN-3B" x="3869" y="0"></use><use xlink:href="#E2714-MJMATHI-3B8" x="4313" y="0"></use><use xlink:href="#E2714-MJMAIN-29" x="4782" y="0"></use></g><g transform="translate(588,-694)"><use xlink:href="#E2714-MJMATHI-62" x="0" y="0"></use><use xlink:href="#E2714-MJMAIN-28" x="429" y="0"></use><g transform="translate(818,0)"><use xlink:href="#E2714-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2714-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2714-MJMAIN-2223" x="2201" y="0"></use><g transform="translate(2756,0)"><use xlink:href="#E2714-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2714-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2714-MJMAIN-29" x="3725" y="0"></use></g></g></g><g transform="translate(6059,0)"><use xlink:href="#E2714-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2714-MJMATHI-74" x="1100" y="-213"></use></g><use xlink:href="#E2714-MJMAIN-2207" x="7193" y="0"></use><g transform="translate(8192,0)"><use xlink:href="#E2714-MJMAIN-6C"></use><use xlink:href="#E2714-MJMAIN-6E" x="278" y="0"></use></g><use xlink:href="#E2714-MJMATHI-3C0" x="9193" y="0"></use><use xlink:href="#E2714-MJMAIN-28" x="9766" y="0"></use><g transform="translate(10155,0)"><use xlink:href="#E2714-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2714-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2714-MJMAIN-2223" x="11538" y="0"></use><g transform="translate(12094,0)"><use xlink:href="#E2714-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2714-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2714-MJMAIN-3B" x="13062" y="0"></use><use xlink:href="#E2714-MJMATHI-3B8" x="13507" y="0"></use><use xlink:href="#E2714-MJMAIN-29" x="13976" y="0"></use><use xlink:href="#E2714-MJSZ3-5D" x="14365" y="-1"></use></g></g></svg></span><script type="math/tex">\displaystyle {\rm E}_b \left[\frac{\pi(A_t \mid S_t;\theta)}{b(A_t \mid S_t)} \Psi_t \nabla\ln\pi(A_t \mid S_t;\theta) \right]</script><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="34.781ex" height="5.797ex" viewBox="0 -1497.2 14974.9 2496" role="img" focusable="false" style="vertical-align: -2.32ex;"><defs><path stroke-width="0" id="E2715-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path><path stroke-width="0" id="E2715-MJMAIN-45" d="M128 619Q121 626 117 628T101 631T58 634H25V680H597V676Q599 670 611 560T625 444V440H585V444Q584 447 582 465Q578 500 570 526T553 571T528 601T498 619T457 629T411 633T353 634Q266 634 251 633T233 622Q233 622 233 621Q232 619 232 497V376H286Q359 378 377 385Q413 401 416 469Q416 471 416 473V493H456V213H416V233Q415 268 408 288T383 317T349 328T297 330Q290 330 286 330H232V196V114Q232 57 237 52Q243 47 289 47H340H391Q428 47 452 50T505 62T552 92T584 146Q594 172 599 200T607 247T612 270V273H652V270Q651 267 632 137T610 3V0H25V46H58Q100 47 109 49T128 61V619Z"></path><path stroke-width="0" id="E2715-MJMATHI-62" d="M73 647Q73 657 77 670T89 683Q90 683 161 688T234 694Q246 694 246 685T212 542Q204 508 195 472T180 418L176 399Q176 396 182 402Q231 442 283 442Q345 442 383 396T422 280Q422 169 343 79T173 -11Q123 -11 82 27T40 150V159Q40 180 48 217T97 414Q147 611 147 623T109 637Q104 637 101 637H96Q86 637 83 637T76 640T73 647ZM336 325V331Q336 405 275 405Q258 405 240 397T207 376T181 352T163 330L157 322L136 236Q114 150 114 114Q114 66 138 42Q154 26 178 26Q211 26 245 58Q270 81 285 114T318 219Q336 291 336 325Z"></path><path stroke-width="0" id="E2715-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2715-MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="0" id="E2715-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2715-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2715-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2715-MJMAIN-2223" d="M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z"></path><path stroke-width="0" id="E2715-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2715-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2715-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2715-MJMAIN-2207" d="M46 676Q46 679 51 683H781Q786 679 786 676Q786 674 617 326T444 -26Q439 -33 416 -33T388 -26Q385 -22 216 326T46 676ZM697 596Q697 597 445 597T193 596Q195 591 319 336T445 80L697 596Z"></path><path stroke-width="0" id="E2715-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2715-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2715-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path><path stroke-width="0" id="E2715-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path><path stroke-width="0" id="E2715-MJSZ3-5B" d="M247 -949V1450H516V1388H309V-887H516V-949H247Z"></path><path stroke-width="0" id="E2715-MJSZ3-5D" d="M11 1388V1450H280V-949H11V-887H218V1388H11Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2715-MJMAIN-3D" x="0" y="0"></use><g transform="translate(1055,0)"><use xlink:href="#E2715-MJMAIN-45" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2715-MJMATHI-62" x="963" y="-213"></use></g><g transform="translate(2306,0)"><use xlink:href="#E2715-MJSZ3-5B"></use><g transform="translate(528,0)"><g transform="translate(120,0)"><rect stroke="none" width="4234" height="60" x="0" y="220"></rect><use xlink:href="#E2715-MJMAIN-31" x="1867" y="676"></use><g transform="translate(60,-694)"><use xlink:href="#E2715-MJMATHI-62" x="0" y="0"></use><use xlink:href="#E2715-MJMAIN-28" x="429" y="0"></use><g transform="translate(818,0)"><use xlink:href="#E2715-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2715-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2715-MJMAIN-2223" x="2064" y="0"></use><g transform="translate(2619,0)"><use xlink:href="#E2715-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2715-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2715-MJMAIN-29" x="3725" y="0"></use></g></g></g><g transform="translate(5002,0)"><use xlink:href="#E2715-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2715-MJMATHI-74" x="1100" y="-213"></use></g><use xlink:href="#E2715-MJMAIN-2207" x="6135" y="0"></use><use xlink:href="#E2715-MJMATHI-3C0" x="6968" y="0"></use><use xlink:href="#E2715-MJMAIN-28" x="7541" y="0"></use><g transform="translate(7930,0)"><use xlink:href="#E2715-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2715-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2715-MJMAIN-2223" x="9313" y="0"></use><g transform="translate(9869,0)"><use xlink:href="#E2715-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2715-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2715-MJMAIN-3B" x="10837" y="0"></use><use xlink:href="#E2715-MJMATHI-3B8" x="11282" y="0"></use><use xlink:href="#E2715-MJMAIN-29" x="11751" y="0"></use><use xlink:href="#E2715-MJSZ3-5D" x="12140" y="-1"></use></g></g></svg></span><script type="math/tex">\displaystyle ={\rm E}_b \left[\frac{1}{b(S_t \mid A_t)} \Psi_t \nabla\pi(A_t \mid S_t;\theta) \right]</script><span> 。这时，更新策略参数 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="1.089ex" height="1.939ex" viewBox="0 -749.6 469 834.7" role="img" focusable="false" style="vertical-align: -0.198ex;"><defs><path stroke-width="0" id="E2716-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2716-MJMATHI-3B8" x="0" y="0"></use></g></svg></span><script type="math/tex">\theta</script><span> 时就应该试图减小 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="28.777ex" height="5.604ex" viewBox="0 -1414.1 12390.1 2412.9" role="img" focusable="false" style="vertical-align: -2.32ex;"><defs><path stroke-width="0" id="E2717-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2717-MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="0" id="E2717-MJMATHI-62" d="M73 647Q73 657 77 670T89 683Q90 683 161 688T234 694Q246 694 246 685T212 542Q204 508 195 472T180 418L176 399Q176 396 182 402Q231 442 283 442Q345 442 383 396T422 280Q422 169 343 79T173 -11Q123 -11 82 27T40 150V159Q40 180 48 217T97 414Q147 611 147 623T109 637Q104 637 101 637H96Q86 637 83 637T76 640T73 647ZM336 325V331Q336 405 275 405Q258 405 240 397T207 376T181 352T163 330L157 322L136 236Q114 150 114 114Q114 66 138 42Q154 26 178 26Q211 26 245 58Q270 81 285 114T318 219Q336 291 336 325Z"></path><path stroke-width="0" id="E2717-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2717-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2717-MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="0" id="E2717-MJMAIN-2223" d="M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z"></path><path stroke-width="0" id="E2717-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2717-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2717-MJMAIN-3A8" d="M340 622Q338 623 335 625T331 629T325 631T314 634T298 635T274 636T239 637H212V683H224Q248 680 389 680T554 683H566V637H539Q479 637 464 635T439 622L438 407Q438 192 439 192Q443 193 449 195T474 207T507 232T536 276T557 344Q560 365 562 417T573 493Q587 536 620 544Q627 546 671 546H715L722 540V515Q714 509 708 509Q680 505 671 476T658 392T644 307Q599 177 451 153L438 151V106L439 61Q446 54 451 52T476 48T539 46H566V0H554Q530 3 389 3T224 0H212V46H239Q259 46 273 46T298 47T314 48T325 51T331 54T335 57T340 61V151Q126 178 117 406Q115 503 69 509Q55 509 55 526Q55 541 59 543T86 546H107H120Q150 546 161 543T184 528Q198 514 204 493Q212 472 213 420T226 316T272 230Q287 216 303 207T330 194L339 192Q340 192 340 407V622Z"></path><path stroke-width="0" id="E2717-MJMAIN-2207" d="M46 676Q46 679 51 683H781Q786 679 786 676Q786 674 617 326T444 -26Q439 -33 416 -33T388 -26Q385 -22 216 326T46 676ZM697 596Q697 597 445 597T193 596Q195 591 319 336T445 80L697 596Z"></path><path stroke-width="0" id="E2717-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2717-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2717-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2717-MJMAIN-2212" x="0" y="0"></use><g transform="translate(778,0)"><g transform="translate(120,0)"><rect stroke="none" width="4234" height="60" x="0" y="220"></rect><use xlink:href="#E2717-MJMAIN-31" x="1867" y="676"></use><g transform="translate(60,-694)"><use xlink:href="#E2717-MJMATHI-62" x="0" y="0"></use><use xlink:href="#E2717-MJMAIN-28" x="429" y="0"></use><g transform="translate(818,0)"><use xlink:href="#E2717-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2717-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2717-MJMAIN-2223" x="2201" y="0"></use><g transform="translate(2756,0)"><use xlink:href="#E2717-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2717-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2717-MJMAIN-29" x="3725" y="0"></use></g></g></g><g transform="translate(5252,0)"><use xlink:href="#E2717-MJMAIN-3A8" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2717-MJMATHI-74" x="1100" y="-213"></use></g><use xlink:href="#E2717-MJMAIN-2207" x="6385" y="0"></use><use xlink:href="#E2717-MJMATHI-3C0" x="7218" y="0"></use><use xlink:href="#E2717-MJMAIN-28" x="7791" y="0"></use><g transform="translate(8180,0)"><use xlink:href="#E2717-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2717-MJMATHI-74" x="1060" y="-213"></use></g><use xlink:href="#E2717-MJMAIN-2223" x="9563" y="0"></use><g transform="translate(10119,0)"><use xlink:href="#E2717-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2717-MJMATHI-74" x="866" y="-213"></use></g><use xlink:href="#E2717-MJMAIN-3B" x="11087" y="0"></use><use xlink:href="#E2717-MJMATHI-3B8" x="11532" y="0"></use><use xlink:href="#E2717-MJMAIN-29" x="12001" y="0"></use></g></svg></span><script type="math/tex">\displaystyle -\frac{1}{b(A_t \mid S_t)} \Psi_t \nabla\pi(A_t \mid S_t;\theta)</script><span> ，具体算法如下：</span></p><div mdtype="math_block" cid="n38" id="mathjax-n38" class="mathjax-block md-end-block md-math-block md-rawblock" spellcheck="false" contenteditable="false"><div class="md-rawblock-container md-math-container" contenteditable="false" tabindex="-1">
						<div class="MathJax_SVG_Display"><span class="MathJax_SVG" id="MathJax-Element-2029-Frame" tabindex="-1" style="font-size: 100%; display: inline-block; zoom: 0.971345;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="101.294ex" height="66.762ex" viewBox="-18.1 -43.5 43612.5 28744.5" role="img" focusable="false" style="vertical-align: -66.661ex; margin-left: -0.042ex; max-width: 100%;"><defs><path stroke-width="0" id="E2791-MJMAINB-38" d="M80 474Q80 561 139 607T278 654Q357 654 411 632Q490 593 494 509Q494 424 416 376L407 371L418 364Q432 356 447 345T481 312T513 260T526 192Q526 100 461 45T285 -11Q184 -11 116 32T48 164Q48 181 50 196T58 225T69 249T84 270T100 286T117 300T134 311T149 321T162 329L152 336Q120 360 100 397T80 474ZM347 404Q404 446 404 503Q404 579 317 599Q309 600 276 600Q178 600 170 538Q170 532 171 527T173 518T178 509T184 501T194 492T205 484T219 476T235 467T254 456T275 445L347 404ZM289 47Q323 47 351 54T402 82T425 137Q425 147 421 161Q411 183 391 197T303 249Q224 293 223 293Q220 291 215 288T197 273T175 248T157 213T149 167Q149 109 188 78T289 47Z"></path><path stroke-width="0" id="E2791-MJMAINB-2D" d="M13 166V278H318V166H13Z"></path><path stroke-width="0" id="E2791-MJMAINB-31" d="M481 0L294 3Q136 3 109 0H96V62H227V304Q227 546 225 546Q169 529 97 529H80V591H97Q231 591 308 647L319 655H333Q355 655 359 644Q361 640 361 351V62H494V0H481Z"></path><path stroke-width="0" id="E2791-MJMAINB-30" d="M266 654H280H282Q500 654 524 418Q529 370 529 320Q529 125 456 52Q397 -10 287 -10Q110 -10 63 154Q45 212 45 316Q45 504 113 585Q140 618 185 636T266 654ZM374 548Q347 604 286 604Q247 604 218 575Q197 552 193 511T188 311Q188 159 196 116Q202 87 225 64T287 41Q339 41 367 87Q379 107 382 152T386 329Q386 518 374 548Z"></path><path stroke-width="0" id="E2791-MJMAINB-2F" d="M451 730Q460 750 479 750Q492 750 501 740T510 718Q508 708 318 244L122 -232Q112 -250 95 -250Q82 -250 73 -241T64 -218Q66 -205 258 261T451 730Z"></path><path stroke-width="0" id="E2791-MJMAIN-22EF" d="M78 250Q78 274 95 292T138 310Q162 310 180 294T199 251Q199 226 182 208T139 190T96 207T78 250ZM525 250Q525 274 542 292T585 310Q609 310 627 294T646 251Q646 226 629 208T586 190T543 207T525 250ZM972 250Q972 274 989 292T1032 310Q1056 310 1074 294T1093 251Q1093 226 1076 208T1033 190T990 207T972 250Z"></path><path stroke-width="0" id="E2791-MJMAIN-38" d="M70 417T70 494T124 618T248 666Q319 666 374 624T429 515Q429 485 418 459T392 417T361 389T335 371T324 363L338 354Q352 344 366 334T382 323Q457 264 457 174Q457 95 399 37T249 -22Q159 -22 101 29T43 155Q43 263 172 335L154 348Q133 361 127 368Q70 417 70 494ZM286 386L292 390Q298 394 301 396T311 403T323 413T334 425T345 438T355 454T364 471T369 491T371 513Q371 556 342 586T275 624Q268 625 242 625Q201 625 165 599T128 534Q128 511 141 492T167 463T217 431Q224 426 228 424L286 386ZM250 21Q308 21 350 55T392 137Q392 154 387 169T375 194T353 216T330 234T301 253T274 270Q260 279 244 289T218 306L210 311Q204 311 181 294T133 239T107 157Q107 98 150 60T250 21Z"></path><path stroke-width="0" id="E2791-MJMAIN-2D" d="M11 179V252H277V179H11Z"></path><path stroke-width="0" id="E2791-MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="0" id="E2791-MJMAIN-32" d="M109 429Q82 429 66 447T50 491Q50 562 103 614T235 666Q326 666 387 610T449 465Q449 422 429 383T381 315T301 241Q265 210 201 149L142 93L218 92Q375 92 385 97Q392 99 409 186V189H449V186Q448 183 436 95T421 3V0H50V19V31Q50 38 56 46T86 81Q115 113 136 137Q145 147 170 174T204 211T233 244T261 278T284 308T305 340T320 369T333 401T340 431T343 464Q343 527 309 573T212 619Q179 619 154 602T119 569T109 550Q109 549 114 549Q132 549 151 535T170 489Q170 464 154 447T109 429Z"></path><path stroke-width="0" id="E2791-MJMAIN-2E" d="M78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60Z"></path><path stroke-width="0" id="E2791-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2791-MJMATHI-62" d="M73 647Q73 657 77 670T89 683Q90 683 161 688T234 694Q246 694 246 685T212 542Q204 508 195 472T180 418L176 399Q176 396 182 402Q231 442 283 442Q345 442 383 396T422 280Q422 169 343 79T173 -11Q123 -11 82 27T40 150V159Q40 180 48 217T97 414Q147 611 147 623T109 637Q104 637 101 637H96Q86 637 83 637T76 640T73 647ZM336 325V331Q336 405 275 405Q258 405 240 397T207 376T181 352T163 330L157 322L136 236Q114 150 114 114Q114 66 138 42Q154 26 178 26Q211 26 245 58Q270 81 285 114T318 219Q336 291 336 325Z"></path><path stroke-width="0" id="E2791-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2791-MJMAIN-22C5" d="M78 250Q78 274 95 292T138 310Q162 310 180 294T199 251Q199 226 182 208T139 190T96 207T78 250Z"></path><path stroke-width="0" id="E2791-MJMAIN-2223" d="M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z"></path><path stroke-width="0" id="E2791-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path><path stroke-width="0" id="E2791-MJMATHI-41" d="M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z"></path><path stroke-width="0" id="E2791-MJMAIN-33" d="M127 463Q100 463 85 480T69 524Q69 579 117 622T233 665Q268 665 277 664Q351 652 390 611T430 522Q430 470 396 421T302 350L299 348Q299 347 308 345T337 336T375 315Q457 262 457 175Q457 96 395 37T238 -22Q158 -22 100 21T42 130Q42 158 60 175T105 193Q133 193 151 175T169 130Q169 119 166 110T159 94T148 82T136 74T126 70T118 67L114 66Q165 21 238 21Q293 21 321 74Q338 107 338 175V195Q338 290 274 322Q259 328 213 329L171 330L168 332Q166 335 166 348Q166 366 174 366Q202 366 232 371Q266 376 294 413T322 525V533Q322 590 287 612Q265 626 240 626Q208 626 181 615T143 592T132 580H135Q138 579 143 578T153 573T165 566T175 555T183 540T186 520Q186 498 172 481T127 463Z"></path><path stroke-width="0" id="E2791-MJMATHI-52" d="M230 637Q203 637 198 638T193 649Q193 676 204 682Q206 683 378 683Q550 682 564 680Q620 672 658 652T712 606T733 563T739 529Q739 484 710 445T643 385T576 351T538 338L545 333Q612 295 612 223Q612 212 607 162T602 80V71Q602 53 603 43T614 25T640 16Q668 16 686 38T712 85Q717 99 720 102T735 105Q755 105 755 93Q755 75 731 36Q693 -21 641 -21H632Q571 -21 531 4T487 82Q487 109 502 166T517 239Q517 290 474 313Q459 320 449 321T378 323H309L277 193Q244 61 244 59Q244 55 245 54T252 50T269 48T302 46H333Q339 38 339 37T336 19Q332 6 326 0H311Q275 2 180 2Q146 2 117 2T71 2T50 1Q33 1 33 10Q33 12 36 24Q41 43 46 45Q50 46 61 46H67Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628Q287 635 230 637ZM630 554Q630 586 609 608T523 636Q521 636 500 636T462 637H440Q393 637 386 627Q385 624 352 494T319 361Q319 360 388 360Q466 361 492 367Q556 377 592 426Q608 449 619 486T630 554Z"></path><path stroke-width="0" id="E2791-MJMAIN-2032" d="M79 43Q73 43 52 49T30 61Q30 68 85 293T146 528Q161 560 198 560Q218 560 240 545T262 501Q262 496 260 486Q259 479 173 263T84 45T79 43Z"></path><path stroke-width="0" id="E2791-MJMATHI-55" d="M107 637Q73 637 71 641Q70 643 70 649Q70 673 81 682Q83 683 98 683Q139 681 234 681Q268 681 297 681T342 682T362 682Q378 682 378 672Q378 670 376 658Q371 641 366 638H364Q362 638 359 638T352 638T343 637T334 637Q295 636 284 634T266 623Q265 621 238 518T184 302T154 169Q152 155 152 140Q152 86 183 55T269 24Q336 24 403 69T501 205L552 406Q599 598 599 606Q599 633 535 637Q511 637 511 648Q511 650 513 660Q517 676 519 679T529 683Q532 683 561 682T645 680Q696 680 723 681T752 682Q767 682 767 672Q767 650 759 642Q756 637 737 637Q666 633 648 597Q646 592 598 404Q557 235 548 205Q515 105 433 42T263 -22Q171 -22 116 34T60 167V183Q60 201 115 421Q164 622 164 628Q164 635 107 637Z"></path><path stroke-width="0" id="E2791-MJMAIN-2190" d="M944 261T944 250T929 230H165Q167 228 182 216T211 189T244 152T277 96T303 25Q308 7 308 0Q308 -11 288 -11Q281 -11 278 -11T272 -7T267 2T263 21Q245 94 195 151T73 236Q58 242 55 247Q55 254 59 257T73 264Q121 283 158 314T215 375T247 434T264 480L267 497Q269 503 270 505T275 509T288 511Q308 511 308 500Q308 493 303 475Q293 438 278 406T246 352T215 315T185 287T165 270H929Q944 261 944 250Z"></path><path stroke-width="0" id="E2791-MJMAIN-2B" d="M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z"></path><path stroke-width="0" id="E2791-MJMATHI-3B3" d="M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z"></path><path stroke-width="0" id="E2791-MJMATHI-71" d="M33 157Q33 258 109 349T280 441Q340 441 372 389Q373 390 377 395T388 406T404 418Q438 442 450 442Q454 442 457 439T460 434Q460 425 391 149Q320 -135 320 -139Q320 -147 365 -148H390Q396 -156 396 -157T393 -175Q389 -188 383 -194H370Q339 -192 262 -192Q234 -192 211 -192T174 -192T157 -193Q143 -193 143 -185Q143 -182 145 -170Q149 -154 152 -151T172 -148Q220 -148 230 -141Q238 -136 258 -53T279 32Q279 33 272 29Q224 -10 172 -10Q117 -10 75 30T33 157ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z"></path><path stroke-width="0" id="E2791-MJMAIN-2C" d="M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z"></path><path stroke-width="0" id="E2791-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2791-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2791-MJMAIN-34" d="M462 0Q444 3 333 3Q217 3 199 0H190V46H221Q241 46 248 46T265 48T279 53T286 61Q287 63 287 115V165H28V211L179 442Q332 674 334 675Q336 677 355 677H373L379 671V211H471V165H379V114Q379 73 379 66T385 54Q393 47 442 46H471V0H462ZM293 211V545L74 212L183 211H293Z"></path><path stroke-width="0" id="E2791-MJMATHI-3B8" d="M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z"></path><path stroke-width="0" id="E2791-MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="0" id="E2791-MJMATHI-49" d="M43 1Q26 1 26 10Q26 12 29 24Q34 43 39 45Q42 46 54 46H60Q120 46 136 53Q137 53 138 54Q143 56 149 77T198 273Q210 318 216 344Q286 624 286 626Q284 630 284 631Q274 637 213 637H193Q184 643 189 662Q193 677 195 680T209 683H213Q285 681 359 681Q481 681 487 683H497Q504 676 504 672T501 655T494 639Q491 637 471 637Q440 637 407 634Q393 631 388 623Q381 609 337 432Q326 385 315 341Q245 65 245 59Q245 52 255 50T307 46H339Q345 38 345 37T342 19Q338 6 332 0H316Q279 2 179 2Q143 2 113 2T65 2T43 1Z"></path><path stroke-width="0" id="E2791-MJMATHI-3C0" d="M132 -11Q98 -11 98 22V33L111 61Q186 219 220 334L228 358H196Q158 358 142 355T103 336Q92 329 81 318T62 297T53 285Q51 284 38 284Q19 284 19 294Q19 300 38 329T93 391T164 429Q171 431 389 431Q549 431 553 430Q573 423 573 402Q573 371 541 360Q535 358 472 358H408L405 341Q393 269 393 222Q393 170 402 129T421 65T431 37Q431 20 417 5T381 -10Q370 -10 363 -7T347 17T331 77Q330 86 330 121Q330 170 339 226T357 318T367 358H269L268 354Q268 351 249 275T206 114T175 17Q164 -11 132 -11Z"></path><path stroke-width="0" id="E2791-MJMATHI-3B1" d="M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z"></path><path stroke-width="0" id="E2791-MJMAIN-2207" d="M46 676Q46 679 51 683H781Q786 679 786 676Q786 674 617 326T444 -26Q439 -33 416 -33T388 -26Q385 -22 216 326T46 676ZM697 596Q697 597 445 597T193 596Q195 591 319 336T445 80L697 596Z"></path><path stroke-width="0" id="E2791-MJMAIN-35" d="M164 157Q164 133 148 117T109 101H102Q148 22 224 22Q294 22 326 82Q345 115 345 210Q345 313 318 349Q292 382 260 382H254Q176 382 136 314Q132 307 129 306T114 304Q97 304 95 310Q93 314 93 485V614Q93 664 98 664Q100 666 102 666Q103 666 123 658T178 642T253 634Q324 634 389 662Q397 666 402 666Q410 666 410 648V635Q328 538 205 538Q174 538 149 544L139 546V374Q158 388 169 396T205 412T256 420Q337 420 393 355T449 201Q449 109 385 44T229 -22Q148 -22 99 32T50 154Q50 178 61 192T84 210T107 214Q132 214 148 197T164 157Z"></path><path stroke-width="0" id="E2791-MJMAIN-5B" d="M118 -250V750H255V710H158V-210H255V-250H118Z"></path><path stroke-width="0" id="E2791-MJMAIN-5D" d="M22 710V750H159V-250H22V-210H119V710H22Z"></path><path stroke-width="0" id="E2791-MJMAIN-36" d="M42 313Q42 476 123 571T303 666Q372 666 402 630T432 550Q432 525 418 510T379 495Q356 495 341 509T326 548Q326 592 373 601Q351 623 311 626Q240 626 194 566Q147 500 147 364L148 360Q153 366 156 373Q197 433 263 433H267Q313 433 348 414Q372 400 396 374T435 317Q456 268 456 210V192Q456 169 451 149Q440 90 387 34T253 -22Q225 -22 199 -14T143 16T92 75T56 172T42 313ZM257 397Q227 397 205 380T171 335T154 278T148 216Q148 133 160 97T198 39Q222 21 251 21Q302 21 329 59Q342 77 347 104T352 209Q352 289 347 316T329 361Q302 397 257 397Z"></path><path stroke-width="0" id="E2791-MJMAIN-37" d="M55 458Q56 460 72 567L88 674Q88 676 108 676H128V672Q128 662 143 655T195 646T364 644H485V605L417 512Q408 500 387 472T360 435T339 403T319 367T305 330T292 284T284 230T278 162T275 80Q275 66 275 52T274 28V19Q270 2 255 -10T221 -22Q210 -22 200 -19T179 0T168 40Q168 198 265 368Q285 400 349 489L395 552H302Q128 552 119 546Q113 543 108 522T98 479L95 458V455H55V458Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><g transform="translate(10646,-2531)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">算</text><g transform="translate(1052,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">法</text></g><use transform="scale(1.2)" xlink:href="#E2791-MJMAINB-38" x="1963" y="0"></use><use transform="scale(1.2)" xlink:href="#E2791-MJMAINB-2D" x="2538" y="0"></use><use transform="scale(1.2)" xlink:href="#E2791-MJMAINB-31" x="2921" y="0"></use><use transform="scale(1.2)" xlink:href="#E2791-MJMAINB-30" x="3496" y="0"></use><g transform="translate(5635,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">异</text></g><g transform="translate(6688,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(7740,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">动</text></g><g transform="translate(8757,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">作</text></g><g transform="translate(9810,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">价</text></g><g transform="translate(10862,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">值</text></g><g transform="translate(11915,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(12968,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(14021,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">者</text></g><use transform="scale(1.2)" xlink:href="#E2791-MJMAINB-2F" x="12561" y="0"></use><g transform="translate(15764,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">评</text></g><g transform="translate(16817,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">论</text></g><g transform="translate(17870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">者</text></g><g transform="translate(18922,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(19975,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" font-weight="bold" stroke="none" transform="scale(49.839) matrix(1 0 0 -1 0 0)">法</text></g></g><g transform="translate(0,-15386)"><g transform="translate(-19,0)"><g transform="translate(0,11324)"><g><rect fill="black" stroke="none" width="1569" height="100" x="0" y="500"></rect></g></g><g transform="translate(0,-11125)"><g><rect fill="black" stroke="none" width="1569" height="100" x="0" y="-500"></rect></g></g></g><g transform="translate(1551,0)"><g transform="translate(0,11324)"><g><rect fill="black" stroke="none" width="41597" height="100" x="0" y="500"></rect></g></g><g transform="translate(0,10024)"><use xlink:href="#E2791-MJMAIN-22EF" x="166" y="0"></use><g transform="translate(2505,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">同</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">算</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">法</text></g><use xlink:href="#E2791-MJMAIN-38" x="2741" y="0"></use><use xlink:href="#E2791-MJMAIN-2D" x="3241" y="0"></use><use xlink:href="#E2791-MJMAIN-31" x="3574" y="0"></use></g><use xlink:href="#E2791-MJMAIN-22EF" x="7746" y="0"></use></g><g transform="translate(0,8724)"><use xlink:href="#E2791-MJMAIN-32"></use><use xlink:href="#E2791-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2791-MJMAIN-32" x="778" y="0"></use><g transform="translate(1278,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2108,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">决</text></g><g transform="translate(2939,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">定</text></g><g transform="translate(3769,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">初</text></g><g transform="translate(4600,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">始</text></g><g transform="translate(5431,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(6261,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g><g transform="translate(7092,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">动</text></g><g transform="translate(7923,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g><g transform="translate(8753,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">对</text></g><g transform="translate(9584,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(10415,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">选</text></g><g transform="translate(11245,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">择</text></g><g transform="translate(12076,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(12907,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g><use xlink:href="#E2791-MJMATHI-53" x="13987" y="0"></use><g transform="translate(14632,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">并</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">用</text></g></g><g transform="translate(17624,0)"><use xlink:href="#E2791-MJMATHI-62" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="429" y="0"></use><use xlink:href="#E2791-MJMAIN-22C5" x="818" y="0"></use><use xlink:href="#E2791-MJMAIN-2223" x="1373" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="1929" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="2574" y="0"></use></g><g transform="translate(20588,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">得</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">到</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">动</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g></g><use xlink:href="#E2791-MJMATHI-41" x="24410" y="0"></use><g transform="translate(25160,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g><g transform="translate(0,7374)"><use xlink:href="#E2791-MJMAIN-32"></use><use xlink:href="#E2791-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2791-MJMAIN-33" x="778" y="0"></use><g transform="translate(1972,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">如</text><g transform="translate(830,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">果</text></g><g transform="translate(1661,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">回</text></g><g transform="translate(2491,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">合</text></g><g transform="translate(3322,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">未</text></g><g transform="translate(4153,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">结</text></g><g transform="translate(4983,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">束</text></g><g transform="translate(5814,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g><g transform="translate(6645,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(7475,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(8306,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">以</text></g><g transform="translate(9137,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">下</text></g><g transform="translate(9967,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">操</text></g><g transform="translate(10798,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g><g transform="translate(11629,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">：</text></g></g></g><g transform="translate(0,6065)"><g transform="translate(2000,0)"><use xlink:href="#E2791-MJMAIN-32"></use><use xlink:href="#E2791-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2791-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2791-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2791-MJMAIN-31" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">采</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">样</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">根</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">据</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g><use xlink:href="#E2791-MJMATHI-53" x="8951" y="0"></use><g transform="translate(9596,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">和</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">动</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">作</text></g></g><use xlink:href="#E2791-MJMATHI-41" x="12588" y="0"></use><g transform="translate(13338,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">得</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">到</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">奖</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">励</text></g></g><use xlink:href="#E2791-MJMATHI-52" x="17160" y="0"></use><g transform="translate(17919,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">和</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">下</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">一</text></g><g transform="translate(2741,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">个</text></g><g transform="translate(3572,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(4403,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g></g><g transform="translate(23403,0)"><use xlink:href="#E2791-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2791-MJMAIN-2032" x="925" y="583"></use></g><g transform="translate(24352,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,4757)"><g transform="translate(2000,0)"><use xlink:href="#E2791-MJMAIN-32"></use><use xlink:href="#E2791-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2791-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2791-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2791-MJMAIN-32" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">执</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">行</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">用</text></g><g transform="translate(6459,0)"><use xlink:href="#E2791-MJMATHI-62" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="429" y="0"></use><use xlink:href="#E2791-MJMAIN-22C5" x="818" y="0"></use><use xlink:href="#E2791-MJMAIN-2223" x="1373" y="0"></use><g transform="translate(1929,0)"><use xlink:href="#E2791-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2791-MJMAIN-2032" x="925" y="583"></use></g><use xlink:href="#E2791-MJMAIN-29" x="2878" y="0"></use></g><g transform="translate(9726,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">得</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">到</text></g></g><g transform="translate(11888,0)"><use xlink:href="#E2791-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2791-MJMAIN-2032" x="1060" y="583"></use></g><g transform="translate(12932,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,3398)"><g transform="translate(2000,0)"><use xlink:href="#E2791-MJMAIN-32"></use><use xlink:href="#E2791-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2791-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2791-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2791-MJMAIN-33" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">估</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">回</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">报</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><use xlink:href="#E2791-MJMATHI-55" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-2190" x="1044" y="0"></use><use xlink:href="#E2791-MJMATHI-52" x="2322" y="0"></use><use xlink:href="#E2791-MJMAIN-2B" x="3303" y="0"></use><use xlink:href="#E2791-MJMATHI-3B3" x="4304" y="0"></use><use xlink:href="#E2791-MJMATHI-71" x="4847" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="5307" y="0"></use><g transform="translate(5696,0)"><use xlink:href="#E2791-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2791-MJMAIN-2032" x="925" y="583"></use></g><use xlink:href="#E2791-MJMAIN-2C" x="6645" y="0"></use><g transform="translate(7089,0)"><use xlink:href="#E2791-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2791-MJMAIN-2032" x="1060" y="583"></use></g><use xlink:href="#E2791-MJMAIN-3B" x="8134" y="0"></use><use xlink:href="#E2791-MJMAINB-77" x="8578" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="9409" y="0"></use></g><g transform="translate(16838,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,1505)"><g transform="translate(2000,0)"><use xlink:href="#E2791-MJMAIN-32"></use><use xlink:href="#E2791-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2791-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2791-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2791-MJMAIN-34" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">策</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">略</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">改</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">进</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><use xlink:href="#E2791-MJMATHI-3B8" x="8951" y="0"></use><g transform="translate(9420,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">以</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">减</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">小</text></g></g><g transform="translate(12412,0)"><use xlink:href="#E2791-MJMAIN-2212" x="0" y="0"></use><g transform="translate(778,0)"><g transform="translate(120,0)"><rect stroke="none" width="3555" height="60" x="0" y="220"></rect><use xlink:href="#E2791-MJMAIN-31" x="1527" y="676"></use><g transform="translate(60,-694)"><use xlink:href="#E2791-MJMATHI-62" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="429" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="818" y="0"></use><use xlink:href="#E2791-MJMAIN-2223" x="1845" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="2401" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="3046" y="0"></use></g></g></g><use xlink:href="#E2791-MJMATHI-49" x="4573" y="0"></use><use xlink:href="#E2791-MJMATHI-71" x="5077" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="5537" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="5926" y="0"></use><use xlink:href="#E2791-MJMAIN-2C" x="6571" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="7016" y="0"></use><use xlink:href="#E2791-MJMAIN-3B" x="7766" y="0"></use><use xlink:href="#E2791-MJMAINB-77" x="8210" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="9041" y="0"></use><use xlink:href="#E2791-MJMATHI-3C0" x="9430" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="10003" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="10392" y="0"></use><use xlink:href="#E2791-MJMAIN-2223" x="11420" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="11976" y="0"></use><use xlink:href="#E2791-MJMAIN-3B" x="12621" y="0"></use><use xlink:href="#E2791-MJMATHI-3B8" x="13066" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="13535" y="0"></use></g><g transform="translate(26336,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g></g></g></g><g transform="translate(0,-1081)"><g transform="translate(6444,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">如</text><g transform="translate(1080,0)"><use xlink:href="#E2791-MJMATHI-3B8" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-2190" x="746" y="0"></use><use xlink:href="#E2791-MJMATHI-3B8" x="2024" y="0"></use><use xlink:href="#E2791-MJMAIN-2B" x="2715" y="0"></use><g transform="translate(3716,0)"><use xlink:href="#E2791-MJMATHI-3B1" x="0" y="0"></use><g transform="translate(640,412)"><use transform="scale(0.707)" xlink:href="#E2791-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2791-MJMATHI-3B8" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2791-MJMAIN-29" x="858" y="0"></use></g></g><g transform="translate(5337,0)"><g transform="translate(120,0)"><rect stroke="none" width="3555" height="60" x="0" y="220"></rect><use xlink:href="#E2791-MJMAIN-31" x="1527" y="676"></use><g transform="translate(60,-694)"><use xlink:href="#E2791-MJMATHI-62" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="429" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="818" y="0"></use><use xlink:href="#E2791-MJMAIN-2223" x="1845" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="2401" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="3046" y="0"></use></g></g></g><use xlink:href="#E2791-MJMATHI-49" x="9133" y="0"></use><use xlink:href="#E2791-MJMATHI-71" x="9637" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="10097" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="10486" y="0"></use><use xlink:href="#E2791-MJMAIN-2C" x="11131" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="11575" y="0"></use><use xlink:href="#E2791-MJMAIN-3B" x="12325" y="0"></use><use xlink:href="#E2791-MJMAINB-77" x="12770" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="13601" y="0"></use><use xlink:href="#E2791-MJMAIN-2207" x="13990" y="0"></use><use xlink:href="#E2791-MJMATHI-3C0" x="14823" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="15396" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="15785" y="0"></use><use xlink:href="#E2791-MJMAIN-2223" x="16813" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="17369" y="0"></use><use xlink:href="#E2791-MJMAIN-3B" x="18014" y="0"></use><use xlink:href="#E2791-MJMATHI-3B8" x="18458" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="18927" y="0"></use></g><g transform="translate(20397,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-3769)"><g transform="translate(4000,0)"><use xlink:href="#E2791-MJMAIN-32"></use><use xlink:href="#E2791-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2791-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2791-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2791-MJMAIN-35" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">价</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">值</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><use xlink:href="#E2791-MJMAINB-77" x="8951" y="0"></use><g transform="translate(9782,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">以</text></g><g transform="translate(1080,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">减</text></g><g transform="translate(1911,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">小</text></g></g><g transform="translate(12774,0)"><g transform="translate(120,0)"><rect stroke="none" width="4613" height="60" x="0" y="220"></rect><g transform="translate(60,693)"><use xlink:href="#E2791-MJMATHI-3C0" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="573" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="962" y="0"></use><use xlink:href="#E2791-MJMAIN-2223" x="1989" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="2545" y="0"></use><use xlink:href="#E2791-MJMAIN-3B" x="3190" y="0"></use><use xlink:href="#E2791-MJMATHI-3B8" x="3635" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="4104" y="0"></use></g><g transform="translate(588,-694)"><use xlink:href="#E2791-MJMATHI-62" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="429" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="818" y="0"></use><use xlink:href="#E2791-MJMAIN-2223" x="1845" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="2401" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="3046" y="0"></use></g></g><use xlink:href="#E2791-MJMAIN-5B" x="4853" y="0"></use><use xlink:href="#E2791-MJMATHI-55" x="5131" y="0"></use><use xlink:href="#E2791-MJMAIN-2212" x="6120" y="0"></use><use xlink:href="#E2791-MJMATHI-71" x="7120" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="7580" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="7969" y="0"></use><use xlink:href="#E2791-MJMAIN-2C" x="8614" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="9059" y="0"></use><use xlink:href="#E2791-MJMAIN-3B" x="9809" y="0"></use><use xlink:href="#E2791-MJMAINB-77" x="10254" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="11085" y="0"></use><g transform="translate(11474,0)"><use xlink:href="#E2791-MJMAIN-5D" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2791-MJMAIN-32" x="393" y="583"></use></g></g><g transform="translate(24979,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">，</text></g></g></g></g><g transform="translate(0,-6457)"><g transform="translate(6444,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">如</text><g transform="translate(1080,0)"><use xlink:href="#E2791-MJMAINB-77" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-2190" x="1108" y="0"></use><use xlink:href="#E2791-MJMAINB-77" x="2386" y="0"></use><use xlink:href="#E2791-MJMAIN-2B" x="3439" y="0"></use><g transform="translate(4440,0)"><use xlink:href="#E2791-MJMATHI-3B1" x="0" y="0"></use><g transform="translate(640,412)"><use transform="scale(0.707)" xlink:href="#E2791-MJMAIN-28" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2791-MJMAINB-77" x="389" y="0"></use><use transform="scale(0.707)" xlink:href="#E2791-MJMAIN-29" x="1220" y="0"></use></g></g><g transform="translate(6317,0)"><g transform="translate(120,0)"><rect stroke="none" width="4613" height="60" x="0" y="220"></rect><g transform="translate(60,693)"><use xlink:href="#E2791-MJMATHI-3C0" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="573" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="962" y="0"></use><use xlink:href="#E2791-MJMAIN-2223" x="1989" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="2545" y="0"></use><use xlink:href="#E2791-MJMAIN-3B" x="3190" y="0"></use><use xlink:href="#E2791-MJMATHI-3B8" x="3635" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="4104" y="0"></use></g><g transform="translate(588,-694)"><use xlink:href="#E2791-MJMATHI-62" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="429" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="818" y="0"></use><use xlink:href="#E2791-MJMAIN-2223" x="1845" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="2401" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="3046" y="0"></use></g></g></g><use xlink:href="#E2791-MJMAIN-5B" x="11170" y="0"></use><use xlink:href="#E2791-MJMATHI-55" x="11448" y="0"></use><use xlink:href="#E2791-MJMAIN-2212" x="12438" y="0"></use><use xlink:href="#E2791-MJMATHI-71" x="13438" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="13898" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="14287" y="0"></use><use xlink:href="#E2791-MJMAIN-2C" x="14932" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="15377" y="0"></use><use xlink:href="#E2791-MJMAIN-3B" x="16127" y="0"></use><use xlink:href="#E2791-MJMAINB-77" x="16571" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="17402" y="0"></use><use xlink:href="#E2791-MJMAIN-5D" x="17791" y="0"></use><use xlink:href="#E2791-MJMAIN-2207" x="18069" y="0"></use><use xlink:href="#E2791-MJMATHI-71" x="18902" y="0"></use><use xlink:href="#E2791-MJMAIN-28" x="19362" y="0"></use><use xlink:href="#E2791-MJMATHI-53" x="19751" y="0"></use><use xlink:href="#E2791-MJMAIN-2C" x="20396" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="20841" y="0"></use><use xlink:href="#E2791-MJMAIN-3B" x="21591" y="0"></use><use xlink:href="#E2791-MJMAINB-77" x="22036" y="0"></use><use xlink:href="#E2791-MJMAIN-29" x="22867" y="0"></use></g><g transform="translate(24336,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-8500)"><g transform="translate(4000,0)"><use xlink:href="#E2791-MJMAIN-32"></use><use xlink:href="#E2791-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2791-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2791-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2791-MJMAIN-36" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">累</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">计</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">折</text></g><g transform="translate(7039,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">扣</text></g><g transform="translate(7870,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(8701,0)"><use xlink:href="#E2791-MJMATHI-49" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-2190" x="781" y="0"></use><use xlink:href="#E2791-MJMATHI-3B3" x="2059" y="0"></use><use xlink:href="#E2791-MJMATHI-49" x="2602" y="0"></use></g><g transform="translate(11807,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">；</text></g></g></g></g><g transform="translate(0,-9825)"><g transform="translate(4000,0)"><use xlink:href="#E2791-MJMAIN-32"></use><use xlink:href="#E2791-MJMAIN-2E" x="500" y="0"></use><use xlink:href="#E2791-MJMAIN-33" x="778" y="0"></use><use xlink:href="#E2791-MJMAIN-2E" x="1278" y="0"></use><use xlink:href="#E2791-MJMAIN-37" x="1556" y="0"></use><g transform="translate(2056,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">（</text></g><g transform="translate(2886,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">更</text></g><g transform="translate(3717,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">新</text></g><g transform="translate(4547,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">状</text></g><g transform="translate(5378,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">态</text></g><g transform="translate(6209,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">）</text></g><g transform="translate(7039,0)"><use xlink:href="#E2791-MJMATHI-53" x="0" y="0"></use><use xlink:href="#E2791-MJMAIN-2190" x="922" y="0"></use><g transform="translate(2200,0)"><use xlink:href="#E2791-MJMATHI-53" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2791-MJMAIN-2032" x="925" y="583"></use></g><use xlink:href="#E2791-MJMAIN-2C" x="3149" y="0"></use><use xlink:href="#E2791-MJMATHI-41" x="3594" y="0"></use><use xlink:href="#E2791-MJMAIN-2190" x="4622" y="0"></use><g transform="translate(5899,0)"><use xlink:href="#E2791-MJMATHI-41" x="0" y="0"></use><use transform="scale(0.707)" xlink:href="#E2791-MJMAIN-2032" x="1060" y="583"></use></g></g><g transform="translate(13984,0)"><g transform="translate(250,0)"><text font-family="STIXGeneral, 'PingFang SC', serif" stroke="none" transform="scale(41.533) matrix(1 0 0 -1 0 0)">。</text></g></g></g></g><g transform="translate(0,-11125)"><g><rect fill="black" stroke="none" width="41597" height="100" x="0" y="-500"></rect></g></g></g></g></g></svg></span></div><script type="math/tex; mode=display" id="MathJax-Element-2029">\; \\ \; \\
\large \textbf{算法 8-10   异策动作价值执行者/评论者算法} \\
\begin{split}
\rule[5pt]{10mm}{0.1em} &\rule[5pt]{265mm}{0.1em} \\
&\cdots \quad \text{同算法 8-1} \quad \cdots \\
&\text{2.2（决定初始状态动作对）选择状态 $S$ ，并用 $b(\cdot \mid S)$ 得到动作 $A$ 。} \\
&\text{2.3 $\;\,$如果回合未结束，执行以下操作：} \\
&\qquad \text{2.3.1（采样）根据状态 $S$ 和动作 $A$ 得到奖励 $R$ 和下一个状态 $S'$ ；} \\
&\qquad \text{2.3.2（执行）用 $b(\cdot \mid S')$ 得到 $A'$ ；} \\
&\qquad \text{2.3.3（估计回报）$U \leftarrow R + \gamma q(S',A';\bold w)$ ；} \\
&\qquad \text{2.3.4（策略改进）更新 $\theta$ 以减小 $-\frac{1}{b(A \mid S)}Iq(S,A;\bold w) \pi(A \mid S; \theta)$ ，} \\
&\qquad \qquad \qquad \;\, \text{如 $\theta \leftarrow \theta + \alpha^{(\theta)} \frac{1}{b(A \mid S)}Iq(S,A;\bold w) \nabla\pi(A \mid S; \theta)$ ；} \\
&\qquad \qquad \text{2.3.5（更新价值）更新 $\bold w$ 以减小 $\frac{\pi(A \mid S; \theta)}{b(A \mid S)}[U - q(S,A;\bold w)]^2$ ，} \\
&\qquad \qquad \qquad \;\, \text{如 $\bold w \leftarrow \bold w + \alpha^{(\bold w)} \frac{\pi(A \mid S; \theta)}{b(A \mid S)} [U - q(S,A;\bold w)] \nabla q(S,A;\bold w)$ ；} \\
&\qquad \qquad \text{2.3.6（更新累计折扣）$I \leftarrow \gamma I$ ；} \\
&\qquad \qquad \text{2.3.7（更新状态）$S \leftarrow S', A \leftarrow A'$ 。} \\
\rule[-5pt]{10mm}{0.1em} &\rule[-5pt]{265mm}{0.1em}
\end{split}
\; \\ \; \\</script>
					</div></div><p><span>Z. Wang 等在文章《Sample efficient actor-critic with experience replay》中提出了</span><strong><span>带经验回放的执行者 / 评论者算法</span></strong><span>（Actor-Critic with Experiment Replay, ACER）。</span></p><p><span>暂略</span></p><h3><a name="五柔性执行者--评论者算法" class="md-header-anchor"></a><span>五、柔性执行者 / 评论者算法</span></h3><p><span>暂略</span></p><h3><a name="六案例双节倒立摆acrobot-v1）" class="md-header-anchor"></a><span>六、案例：双节倒立摆（Acrobot-v1）</span></h3><p><span>本节使用 gym 库中的双节倒立摆（Acrobot-v1）作为实验对象。该对象由两节杆子连接组成，并将其中一根杆子未连接的一端固定住，环境的状态和动作取值在</span><a href='https://github.com/openai/gym/blob/master/gym/envs/classic_control/acrobot.py' target='_blank' title='Acrobot-v1'><span>源代码</span></a><span>中都有说明；奖励则是每执行一个动作返回 -1 ，回合结束时的奖励值为 0 ；当回合步数达到 500 或活动端（未固定的那节杆子）在垂直方向上高于原点（固定点）至少一节杆子的高度时，该回合结束；目的是以尽可能少的步数结束回合。另外，该对象的动力学方程式已知的，但非常复杂，即使知道该表达式，也不可能求出最优动作的闭式解。</span></p><p><span>代码中的智能体类 </span><code>QAC</code><span> 实现了动作价值执行者 / 评论者类，与书中代码基本一致。 </span><code>AdvantageAC</code><span> 实现了优势执行者 / 评论者类，稍微改写了下，使之继承于 </span><code>QAC</code><span> 。</span><code>ElibilityTraceAC</code><span> 实现了带资格迹的优势执行者 / 评论者类，但书中源代码在计算评论者的资格迹时，可能是由复制粘贴造成的疏忽，将 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="9.329ex" height="2.71ex" viewBox="0 -832.7 4016.7 1166.9" role="img" focusable="false" style="vertical-align: -0.776ex;"><defs><path stroke-width="0" id="E2718-MJMAIN-2207" d="M46 676Q46 679 51 683H781Q786 679 786 676Q786 674 617 326T444 -26Q439 -33 416 -33T388 -26Q385 -22 216 326T46 676ZM697 596Q697 597 445 597T193 596Q195 591 319 336T445 80L697 596Z"></path><path stroke-width="0" id="E2718-MJMATHI-76" d="M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z"></path><path stroke-width="0" id="E2718-MJMAIN-28" d="M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z"></path><path stroke-width="0" id="E2718-MJMATHI-53" d="M308 24Q367 24 416 76T466 197Q466 260 414 284Q308 311 278 321T236 341Q176 383 176 462Q176 523 208 573T273 648Q302 673 343 688T407 704H418H425Q521 704 564 640Q565 640 577 653T603 682T623 704Q624 704 627 704T632 705Q645 705 645 698T617 577T585 459T569 456Q549 456 549 465Q549 471 550 475Q550 478 551 494T553 520Q553 554 544 579T526 616T501 641Q465 662 419 662Q362 662 313 616T263 510Q263 480 278 458T319 427Q323 425 389 408T456 390Q490 379 522 342T554 242Q554 216 546 186Q541 164 528 137T492 78T426 18T332 -20Q320 -22 298 -22Q199 -22 144 33L134 44L106 13Q83 -14 78 -18T65 -22Q52 -22 52 -14Q52 -11 110 221Q112 227 130 227H143Q149 221 149 216Q149 214 148 207T144 186T142 153Q144 114 160 87T203 47T255 29T308 24Z"></path><path stroke-width="0" id="E2718-MJMAIN-3B" d="M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z"></path><path stroke-width="0" id="E2718-MJMAINB-77" d="M624 444Q636 441 722 441Q797 441 800 444H805V382H741L593 11Q592 10 590 8T586 4T584 2T581 0T579 -2T575 -3T571 -3T567 -4T561 -4T553 -4H542Q525 -4 518 6T490 70Q474 110 463 137L415 257L367 137Q357 111 341 72Q320 17 313 7T289 -4H277Q259 -4 253 -2T238 11L90 382H25V444H32Q47 441 140 441Q243 441 261 444H270V382H222L310 164L382 342L366 382H303V444H310Q322 441 407 441Q508 441 523 444H531V382H506Q481 382 481 380Q482 376 529 259T577 142L674 382H617V444H624Z"></path><path stroke-width="0" id="E2718-MJMAIN-29" d="M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2718-MJMAIN-2207" x="0" y="0"></use><use xlink:href="#E2718-MJMATHI-76" x="833" y="0"></use><use xlink:href="#E2718-MJMAIN-28" x="1318" y="0"></use><use xlink:href="#E2718-MJMATHI-53" x="1707" y="0"></use><use xlink:href="#E2718-MJMAIN-3B" x="2352" y="0"></use><use xlink:href="#E2718-MJMAINB-77" x="2796" y="0"></use><use xlink:href="#E2718-MJMAIN-29" x="3627" y="0"></use></g></svg></span><script type="math/tex">\nabla v(S;\bold w)</script><span> 乘以了折扣 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="1.171ex" height="1.842ex" viewBox="0 -749.6 504 793.1" role="img" focusable="false" style="vertical-align: -0.101ex;"><defs><path stroke-width="0" id="E2720-MJMATHI-49" d="M43 1Q26 1 26 10Q26 12 29 24Q34 43 39 45Q42 46 54 46H60Q120 46 136 53Q137 53 138 54Q143 56 149 77T198 273Q210 318 216 344Q286 624 286 626Q284 630 284 631Q274 637 213 637H193Q184 643 189 662Q193 677 195 680T209 683H213Q285 681 359 681Q481 681 487 683H497Q504 676 504 672T501 655T494 639Q491 637 471 637Q440 637 407 634Q393 631 388 623Q381 609 337 432Q326 385 315 341Q245 65 245 59Q245 52 255 50T307 46H339Q345 38 345 37T342 19Q338 6 332 0H316Q279 2 179 2Q143 2 113 2T65 2T43 1Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2720-MJMATHI-49" x="0" y="0"></use></g></svg></span><script type="math/tex">I</script><span> ；根据算法 8-4 ，折扣 </span><span class="MathJax_SVG" tabindex="-1" style="font-size: 100%; display: inline-block;"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="1.171ex" height="1.842ex" viewBox="0 -749.6 504 793.1" role="img" focusable="false" style="vertical-align: -0.101ex;"><defs><path stroke-width="0" id="E2720-MJMATHI-49" d="M43 1Q26 1 26 10Q26 12 29 24Q34 43 39 45Q42 46 54 46H60Q120 46 136 53Q137 53 138 54Q143 56 149 77T198 273Q210 318 216 344Q286 624 286 626Q284 630 284 631Q274 637 213 637H193Q184 643 189 662Q193 677 195 680T209 683H213Q285 681 359 681Q481 681 487 683H497Q504 676 504 672T501 655T494 639Q491 637 471 637Q440 637 407 634Q393 631 388 623Q381 609 337 432Q326 385 315 341Q245 65 245 59Q245 52 255 50T307 46H339Q345 38 345 37T342 19Q338 6 332 0H316Q279 2 179 2Q143 2 113 2T65 2T43 1Z"></path></defs><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"><use xlink:href="#E2720-MJMATHI-49" x="0" y="0"></use></g></svg></span><script type="math/tex">I</script><span> 只出现在行动者的资格迹计算中，因此将该操作按算法纠正了；不过经简单的两次测试发现，是否纠正似乎对结果影响并不明显。具体代码此处不再展示。</span></p><p><span>其他算法暂略</span></p><p>&nbsp;</p></div>
</body>
</html>